2024-04-26T21:07:27.359 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:07:27.360 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206 --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/orig.config.yaml 2024-04-26T21:07:27.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:07:27.507 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7675206' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhef9fIanZyyV04PnAodD9TchsY1YKiHpcf6kEUBcPL+SWPNqmF2wNXeom1al4IEBjytX0a8lhXC8BGs3mQRL4= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfTqABpds1YpiUBt1a3TRkE0TRIUCc+Jfg9fL1sHFkOCexV+PaFb1JsKJgTPkUkhHWIJ+F4pRJyybjC4Amrjog= smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6T4C3cqgJiptYm9ScPY5OGgfiouzKH3nMH6DZwiitrHp5vaFGjXZ6naTAm8/fTs0cppSQC8p4PSX2pL/aCgUA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:07:27.508 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:07:27.509 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:07:27.509 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:07:27.511 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:07:27.539 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:07:27.539 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:07:27.539 INFO:teuthology.packaging:ref: None 2024-04-26T21:07:27.539 INFO:teuthology.packaging:tag: None 2024-04-26T21:07:27.540 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:07:27.540 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:27.540 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:07:27.757 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:07:27.759 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:07:27.765 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:07:27.765 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:07:27.773 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:07:27.794 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:07:27.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:07:27.822 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.289075', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhef9fIanZyyV04PnAodD9TchsY1YKiHpcf6kEUBcPL+SWPNqmF2wNXeom1al4IEBjytX0a8lhXC8BGs3mQRL4='} 2024-04-26T21:07:27.840 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.285440', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfTqABpds1YpiUBt1a3TRkE0TRIUCc+Jfg9fL1sHFkOCexV+PaFb1JsKJgTPkUkhHWIJ+F4pRJyybjC4Amrjog='} 2024-04-26T21:07:27.863 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.287480', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6T4C3cqgJiptYm9ScPY5OGgfiouzKH3nMH6DZwiitrHp5vaFGjXZ6naTAm8/fTs0cppSQC8p4PSX2pL/aCgUA='} 2024-04-26T21:07:27.863 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:07:27.869 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T21:07:27.869 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T21:07:27.869 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T21:07:27.869 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:07:27.970 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fae34d7e160>, signals=[15]) 2024-04-26T21:07:27.971 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:07:27.976 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:07:27.976 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:07:27.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:07:28.055 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-04-26T21:07:28.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:07:28.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-04-26T21:07:28.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:07:28.206 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:07:28.212 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-26T21:07:28.231 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-26T21:07:28.231 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-26T21:07:28.288 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:07:28.289 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:07:28.290 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-26T21:07:28.314 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-04-26T21:07:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-04-26T21:07:28.332 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:07:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:07:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:07:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:07:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:07:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:07:28.390 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-04-26T21:07:28.414 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-04-26T21:07:28.432 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-04-26T21:07:28.432 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-04-26T21:07:28.489 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-04-26T21:07:28.489 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:07:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:07:28.491 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-04-26T21:07:28.505 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:07:28.515 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:07:28.522 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:07:28.522 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:07:28.524 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:07:28.527 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:07:28.546 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:07:28.551 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:07:28.552 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:07:28.581 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:07:28.584 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:07:28.696 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:07:28.757 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:07:28.772 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:07:28.772 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T21:07:28.772 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:07:28.773 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:07:28.773 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:07:28.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:28.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:28.790 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.790 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:07:28.790 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-04-26T21:07:28.791 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.791 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:07:28.791 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-04-26T21:07:28.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:28.792 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.792 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:07:28.792 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-04-26T21:07:28.848 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:07:28.848 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.848 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:07:28.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.853 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:07:28.853 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:07:28.853 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.853 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:07:28.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.854 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.854 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:07:28.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:28.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum upgrade -y linux-firmware 2024-04-26T21:07:29.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2024-04-26T21:07:29.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum upgrade -y linux-firmware 2024-04-26T21:07:31.205 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T21:07:31.758 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.0 MB 00:00 2024-04-26T21:07:32.306 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-04-26T21:07:33.734 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-26T21:07:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-26T21:07:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-04-26T21:07:39.268 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-26T21:07:39.465 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-26T21:07:39.915 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-26T21:07:40.710 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-26T21:07:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T21:07:41.591 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-26T21:07:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T21:07:42.928 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T21:07:43.287 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T21:07:46.962 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-26T21:07:47.975 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T21:07:48.392 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T21:07:48.961 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-04-26T21:07:48.962 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-04-26T21:07:48.962 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-04-26T21:07:49.071 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T21:07:49.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:49.287 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:07:49.287 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:07:49.289 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi143.front.sepia.ceph.com to complete... 2024-04-26T21:07:49.929 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-26T21:07:49.930 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-04-26T21:07:49.931 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-26T21:07:50.025 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:07:50.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:50.165 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:07:50.165 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:07:50.168 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-04-26T21:07:50.368 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-26T21:07:50.369 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-04-26T21:07:50.369 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-26T21:07:50.465 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:07:50.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:07:50.629 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:07:50.629 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:07:50.635 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi111.front.sepia.ceph.com to complete... 2024-04-26T21:08:07.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26 21:08:07 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:08:07.133 INFO:teuthology.orchestra.run.smithi143.stderr:FINISHED --2024-04-26 21:08:07-- 2024-04-26T21:08:07.133 INFO:teuthology.orchestra.run.smithi143.stderr:Total wall clock time: 18s 2024-04-26T21:08:07.133 INFO:teuthology.orchestra.run.smithi143.stderr:Downloaded: 1 files, 341M in 18s (19.5 MB/s) 2024-04-26T21:08:07.137 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T21:08:07.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:08:09.283 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying... ######################################## 2024-04-26T21:08:09.423 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing... ######################################## 2024-04-26T21:08:09.517 INFO:teuthology.orchestra.run.smithi143.stdout:Updating / installing... 2024-04-26T21:08:17.510 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:08:20.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26 21:08:20 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:08:20.109 INFO:teuthology.orchestra.run.smithi111.stderr:FINISHED --2024-04-26 21:08:20-- 2024-04-26T21:08:20.109 INFO:teuthology.orchestra.run.smithi111.stderr:Total wall clock time: 29s 2024-04-26T21:08:20.109 INFO:teuthology.orchestra.run.smithi111.stderr:Downloaded: 1 files, 341M in 29s (11.7 MB/s) 2024-04-26T21:08:20.113 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:08:20.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:08:22.189 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying... ######################################## 2024-04-26T21:08:22.318 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing... ######################################## 2024-04-26T21:08:22.440 INFO:teuthology.orchestra.run.smithi111.stdout:Updating / installing... 2024-04-26T21:08:24.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26 21:08:24 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:08:24.886 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-04-26 21:08:24-- 2024-04-26T21:08:24.886 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 35s 2024-04-26T21:08:24.886 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 341M in 34s (9.97 MB/s) 2024-04-26T21:08:24.889 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:08:24.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:08:27.012 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2024-04-26T21:08:27.226 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2024-04-26T21:08:27.326 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2024-04-26T21:08:30.418 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:08:35.244 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:08:56.917 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:08:56.917 DEBUG:teuthology.orchestra.run.smithi143:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:08:57.003 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:08:57.004 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:08:57.005 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:08:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:08:57.007 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:08:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:08:57.009 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:08:57.010 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:08:57.011 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:08:57.012 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:08:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:08:57.014 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:08:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:08:57.016 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:08:57.017 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:08:57.018 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:08:57.019 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:08:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:08:57.021 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:08:57.022 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:08:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:08:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:08:57.025 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:08:57.026 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:08:57.027 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:08:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:08:57.029 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:08:57.030 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:08:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:08:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:08:57.033 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:08:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:08:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:08:57.036 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:08:57.037 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:08:57.038 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:08:57.039 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:08:57.040 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:08:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:08:57.042 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:08:57.043 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:08:57.044 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:08:57.045 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:08:57.046 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:08:57.047 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:08:57.048 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:08:57.049 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:08:57.050 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:08:57.051 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:08:57.052 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:08:57.053 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:08:57.054 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:08:57.055 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:08:57.056 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:08:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:08:57.058 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:08:57.059 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:08:57.060 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:08:57.061 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:08:57.062 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:08:57.063 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:08:57.064 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:08:57.065 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:08:57.066 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:08:57.067 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:08:57.068 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:08:57.069 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:08:57.070 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:08:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:08:57.073 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:08:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:08:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:08:57.076 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:08:57.077 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:08:57.078 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:08:57.079 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:08:57.080 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:08:57.081 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:08:57.082 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:08:57.083 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:08:57.084 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:08:57.085 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:08:57.086 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:08:57.087 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:08:57.088 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:08:57.089 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:08:57.090 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:08:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:08:57.092 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:08:57.093 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:08:57.094 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:08:57.095 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:08:57.096 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:08:57.097 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:08:57.098 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:08:57.099 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:08:57.100 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:08:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:08:57.102 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:08:57.103 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:08:57.104 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:08:57.105 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:08:57.106 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:08:57.107 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:08:57.108 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:08:57.109 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:08:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:08:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:08:57.112 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:08:57.113 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:08:57.114 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:08:57.115 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:08:57.116 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:08:57.117 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:08:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:08:57.119 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:08:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:08:57.121 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:08:57.122 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:08:57.123 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:08:57.124 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:08:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:08:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:08:57.127 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:08:57.128 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:08:57.129 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:08:57.130 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:08:57.131 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:08:57.132 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:08:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:08:57.134 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:08:57.135 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:08:57.136 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:08:57.137 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:08:57.138 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:08:57.139 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:08:57.140 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:08:57.140 INFO:teuthology.orchestra.run.smithi143.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:08:57.143 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.144 DEBUG:teuthology.orchestra.run.smithi143:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout: fi 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:done 2024-04-26T21:08:57.167 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.168 INFO:teuthology.orchestra.run.smithi143.stdout:fi 2024-04-26T21:08:57.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-04-26T21:08:57.242 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-04-26T21:08:57.242 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 68.el9 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8264254 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-04-26T21:08:57.243 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:08:57.244 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:08:57.245 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:08:57.245 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:57.245 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:08:57.821 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-04-26T21:08:59.175 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:08:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-04-26T21:08:59.211 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-04-26T21:08:59.249 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.6mM1Q0d4cS 2024-04-26T21:08:59.249 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6mM1Q0d4cS 2024-04-26T21:08:59.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.6mM1Q0d4cS 2024-04-26T21:08:59.459 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.6mM1Q0d4cS is 6KB 2024-04-26T21:08:59.479 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.6mM1Q0d4cS 2024-04-26T21:08:59.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:08:59.554 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:08:59.554 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:08:59.555 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:08:59.555 INFO:teuthology.orchestra.run.smithi143.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:08:59.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:08:59.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-04-26T21:09:07.909 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:09:07.909 DEBUG:teuthology.orchestra.run.smithi111:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:09:07.985 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:09:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:09:07.987 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:09:07.988 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:09:07.989 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:09:07.990 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:09:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:09:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:09:07.993 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:09:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:09:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:09:07.996 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:09:07.997 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:09:07.998 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:09:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:09:08.000 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:09:08.001 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:09:08.002 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:09:08.003 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:09:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:09:08.005 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:09:08.006 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:09:08.007 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:09:08.008 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:09:08.009 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:09:08.010 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:09:08.011 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:09:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:09:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:09:08.015 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:09:08.016 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:09:08.017 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:09:08.018 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:09:08.019 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:09:08.020 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:09:08.021 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:09:08.022 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:09:08.023 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:09:08.024 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:09:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:09:08.026 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:09:08.027 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:09:08.028 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:09:08.029 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:09:08.030 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:09:08.031 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:09:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:09:08.033 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:09:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:09:08.035 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:09:08.036 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:09:08.037 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:09:08.038 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:09:08.039 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:09:08.040 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:09:08.043 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:09:08.045 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:09:08.046 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:09:08.047 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:09:08.048 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:09:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:09:08.050 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:09:08.051 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:09:08.052 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:09:08.053 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:09:08.054 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:09:08.055 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:09:08.056 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:09:08.057 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:09:08.058 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:09:08.059 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:09:08.060 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:09:08.061 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:09:08.062 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:09:08.063 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:09:08.064 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:09:08.065 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:09:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:09:08.067 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:09:08.068 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:09:08.069 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:09:08.124 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:08.124 DEBUG:teuthology.orchestra.run.smithi111:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:09:08.148 INFO:teuthology.orchestra.run.smithi111.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: fi 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:done 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-26T21:09:08.149 INFO:teuthology.orchestra.run.smithi111.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:08.150 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-04-26T21:09:08.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-04-26T21:09:08.225 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 68.el9 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8264254 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:09:08.226 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:09:08.227 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:09:08.228 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:09:08.228 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:08.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:09:08.785 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-04-26T21:09:10.170 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:09:10.195 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-04-26T21:09:10.198 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-04-26T21:09:10.212 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.iuwoZWl9Do 2024-04-26T21:09:10.213 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iuwoZWl9Do 2024-04-26T21:09:10.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.iuwoZWl9Do 2024-04-26T21:09:10.413 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.iuwoZWl9Do is 6KB 2024-04-26T21:09:10.460 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.iuwoZWl9Do 2024-04-26T21:09:10.474 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:09:10.540 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:09:10.540 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:09:10.540 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:09:10.540 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:09:10.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:10.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-04-26T21:09:12.146 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:09:12.146 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:09:12.217 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:09:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:09:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:09:12.220 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:09:12.221 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:09:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:09:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:09:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:09:12.225 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:09:12.226 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:09:12.227 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:09:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:09:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:09:12.230 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:09:12.231 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:09:12.232 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:09:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:09:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:09:12.235 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:09:12.236 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:09:12.237 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:09:12.238 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:09:12.239 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:09:12.240 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:09:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:09:12.242 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:09:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:09:12.244 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:09:12.245 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:09:12.246 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:09:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:09:12.248 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:09:12.249 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:09:12.250 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:09:12.251 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:09:12.252 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:09:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:09:12.254 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:09:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:09:12.256 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:09:12.257 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:09:12.258 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:09:12.259 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:09:12.260 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:09:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:09:12.262 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:09:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:09:12.264 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:09:12.265 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:09:12.266 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:09:12.267 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:09:12.268 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:09:12.269 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:09:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:09:12.271 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:09:12.272 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:09:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:09:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:09:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:09:12.276 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:09:12.277 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:09:12.278 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:09:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:09:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:09:12.281 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:09:12.282 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:09:12.283 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:09:12.284 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:09:12.285 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:09:12.286 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:09:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:09:12.288 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:09:12.289 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:09:12.290 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:09:12.291 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:09:12.292 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:09:12.293 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:09:12.294 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:09:12.295 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:09:12.296 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:09:12.297 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:09:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:09:12.299 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:09:12.300 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:09:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:09:12.302 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:09:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:09:12.304 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:09:12.305 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:09:12.306 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:09:12.307 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:09:12.308 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:09:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:09:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:09:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:09:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:09:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:09:12.314 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:09:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:09:12.316 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:09:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:09:12.318 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:09:12.319 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:09:12.320 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:09:12.321 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:09:12.322 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:09:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:09:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:09:12.325 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:09:12.326 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:09:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:09:12.328 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:09:12.329 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:09:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:09:12.331 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:09:12.332 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:09:12.333 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:09:12.334 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:09:12.335 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:09:12.336 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:09:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:09:12.338 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:09:12.339 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:09:12.340 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:09:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:09:12.342 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:09:12.343 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:09:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:09:12.345 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:09:12.346 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:09:12.347 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:09:12.348 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:09:12.349 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:09:12.350 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:09:12.354 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.354 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout: fi 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-04-26T21:09:12.376 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-04-26T21:09:12.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-04-26T21:09:12.459 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:09:12.460 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:09:12.461 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-04-26T21:09:12.461 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:09:12.461 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:09:12.461 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:09:12.462 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:12.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:09:12.995 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-04-26T21:09:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:09:14.398 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-04-26T21:09:14.401 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-04-26T21:09:14.432 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.l1F8EESBOg 2024-04-26T21:09:14.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.l1F8EESBOg 2024-04-26T21:09:14.497 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.l1F8EESBOg 2024-04-26T21:09:14.638 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.l1F8EESBOg is 6KB 2024-04-26T21:09:14.687 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.l1F8EESBOg 2024-04-26T21:09:14.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:09:14.771 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:09:14.771 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:09:14.771 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:09:14.771 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:09:14.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:09:14.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-04-26T21:09:29.758 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-26T21:09:29.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:09:29.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:40.701 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T21:09:40.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:09:40.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:44.926 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T21:09:44.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:09:44.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:48.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-04-26T21:09:57.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:09:57.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:09:59.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-26T21:10:00.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-04-26T21:10:03.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-26T21:10:08.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:10:08.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:11.266 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-26T21:10:12.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:10:12.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:12.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:10:12.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:15.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-26T21:10:23.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:10:23.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:26.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-26T21:10:27.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:10:27.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:30.690 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-04-26T21:10:41.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:10:41.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:45.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:10:45.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:10:45.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-26T21:10:46.844 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-04-26T21:10:47.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:10:47.495 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:10:47.495 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-04-26T21:10:47.534 INFO:teuthology.orchestra.run.smithi143.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:10:47.535 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:10:47.535 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:10:47.535 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:10:48.536 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-26T21:10:48.536 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:10:48.746 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2024-04-26T21:10:48.797 DEBUG:teuthology.parallel:result is None 2024-04-26T21:10:59.874 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-04-26T21:11:00.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:11:00.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:11:01.253 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-26T21:11:01.832 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:11:01.832 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:11:01.832 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-04-26T21:11:01.871 INFO:teuthology.orchestra.run.smithi097.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:11:01.871 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:11:01.871 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:11:01.871 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:11:02.872 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T21:11:02.873 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:11:02.989 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-04-26T21:11:03.031 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:17.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:11:17.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:11:18.314 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-26T21:11:18.900 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:11:18.900 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:11:18.900 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-04-26T21:11:18.942 INFO:teuthology.orchestra.run.smithi111.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:11:18.942 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:11:18.942 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:11:18.942 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:11:19.943 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T21:11:19.943 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:11:20.160 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2024-04-26T21:11:20.224 DEBUG:teuthology.parallel:result is None 2024-04-26T21:11:20.225 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:11:20.232 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:11:20.232 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:11:20.241 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:11:20.249 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:11:20.309 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:11:20.316 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:11:20.323 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:11:20.323 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:11:20.330 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:11:20.335 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:11:20.412 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:11:20.419 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:11:20.419 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:11:20.427 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:11:20.431 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:11:20.513 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.516 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.532 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.542 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.549 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.563 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:11:20.567 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:11:20.575 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:11:20.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:11:20.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:11:20.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:11:20.668 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:11:20.678 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:11:20.678 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:11:20.682 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:11:20.687 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:11:20.739 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:20.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:20.916 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:20.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:21.080 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:11:21.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:11:21.164 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:21.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:21.338 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:21.415 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:21.489 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:11:21.489 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:11:21.553 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:21.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:11:21.738 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:21.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:11:21.906 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:11:21.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:11:21.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-26T21:11:21.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-04-26T21:11:22.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-04-26T21:11:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:11:22.109 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:11:22.148 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:11:22.647 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:11:22.655 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:11:22.655 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:11:22.666 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:11:22.674 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:11:22.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-26T21:11:22.807 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-26T21:11:22.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-04-26T21:11:22.940 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-04-26T21:11:22.944 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-04-26T21:11:23.077 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-04-26T21:11:23.082 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:11:23.082 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-26T21:11:23.138 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-26T21:11:23.138 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-04-26T21:11:23.189 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-04-26T21:11:23.190 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-04-26T21:11:23.245 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-04-26T21:11:23.246 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:11:23.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:11:23.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:23.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:11:23.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:23.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:11:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:23.450 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:11:23.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:11:23.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:11:23.594 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:11:23.669 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:11:23.679 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T21:11:23.684 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T21:11:23.833 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:11:23.851 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:11:23.853 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:16:29.620 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2024-04-26T21:16:29.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:16:29.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:16:29.735 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-26T21:16:29.884 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T21:16:29.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:16:29.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:16:29.993 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-26T21:16:30.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-26T21:16:30.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:16:30.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:16:30.270 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-04-26T21:16:30.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-04-26T21:16:30.423 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:16:30.436 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:16:30.436 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:16:30.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:16:30.444 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:16:30.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:16:30.452 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:16:30.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:16:30.529 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:16:30.539 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:16:30.544 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:16:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:16:30.580 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:16:30.587 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:16:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-26T21:16:30.657 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-04-26T21:16:30.657 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-04-26T21:16:30.671 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-26T21:16:30.685 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-04-26T21:16:30.689 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-04-26T21:16:30.709 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:16:30.724 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:16:30.729 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:16:30.749 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:16:30.766 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:16:30.770 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:16:30.891 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.900 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.903 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:16:30.910 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.912 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.918 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:16:30.919 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:16:30.928 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:16:30.928 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:16:30.929 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:16:30.929 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:16:30.934 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:16:30.934 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:16:30.935 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:16:30.935 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:16:30.937 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:16:30.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:16:30.938 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:16:30.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:16:31.080 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:16:31.080 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:16:31.084 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:16:31.084 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:16:31.208 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:16:31.208 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:16:31.307 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:16:31.307 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:16:31.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:16:31.322 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:16:31.322 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:16:31.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:16:31.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:16:31.395 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:16:31.411 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:16:31.412 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:16:31.439 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:16:31.439 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:16:31.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:16:31.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:16:31.527 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:16:31.528 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:16:31.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:16:31.636 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:16:31.650 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:16:31.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:16:31.713 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-04-26T21:16:31.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-04-26T21:16:31.729 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-04-26T21:16:31.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-04-26T21:16:31.764 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:16:31.844 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-26T21:16:31.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-26T21:16:32.031 INFO:teuthology.orchestra.run.smithi111.stdout:48 files removed 2024-04-26T21:16:32.047 INFO:teuthology.orchestra.run.smithi143.stdout:48 files removed 2024-04-26T21:16:32.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:16:32.086 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:16:32.173 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-26T21:16:32.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:16:33.141 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 131 kB/s | 84 kB 00:00 2024-04-26T21:16:33.219 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 121 kB/s | 84 kB 00:00 2024-04-26T21:16:33.276 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 132 kB/s | 84 kB 00:00 2024-04-26T21:16:33.683 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-26T21:16:33.711 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-26T21:16:33.778 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-26T21:16:34.170 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T21:16:34.193 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T21:16:34.295 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T21:16:34.760 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T21:16:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-04-26T21:16:38.100 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-26T21:16:38.453 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-26T21:16:43.254 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-04-26T21:16:43.275 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 926 kB/s | 8.0 MB 00:08 2024-04-26T21:16:44.937 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-26T21:16:46.410 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 19 MB 00:01 2024-04-26T21:16:48.639 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 21 MB 00:03 2024-04-26T21:16:49.633 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 833 kB/s | 6.2 MB 00:07 2024-04-26T21:16:51.491 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-26T21:16:51.492 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-26T21:16:52.694 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-26T21:16:53.298 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-26T21:16:54.564 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T21:16:55.122 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T21:16:56.955 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:16:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:16:56.977 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:16:56.978 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:16:56.979 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:16:56.981 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:16:56.983 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:16:56.986 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:16:56.988 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:16:57.059 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:16:57.094 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:16:57.095 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:16:57.096 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:16:57.097 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:16:57.098 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:16:57.099 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:16:57.100 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:16:57.101 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:16:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:16:57.103 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:16:57.104 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-26T21:16:57.105 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T21:16:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-26T21:16:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-26T21:16:57.106 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:16:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 222 M 2024-04-26T21:16:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-26T21:16:58.549 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-26T21:16:58.635 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-26T21:16:58.740 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.5 MB/s | 889 kB 00:00 2024-04-26T21:16:58.791 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.9 MB/s | 145 kB 00:00 2024-04-26T21:16:59.049 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-04-26T21:16:59.241 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-04-26T21:16:59.467 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-04-26T21:16:59.725 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-04-26T21:17:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 17 MB 00:01 2024-04-26T21:17:00.996 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T21:17:01.100 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:17:01.101 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:17:01.121 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:17:01.123 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:17:01.124 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:17:01.125 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:17:01.128 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:17:01.131 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:17:01.132 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:17:01.206 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-26T21:17:01.217 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 7.2 MB/s | 21 MB 00:02 2024-04-26T21:17:01.238 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:17:01.239 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:17:01.240 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:17:01.241 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:17:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:17:01.243 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:17:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:17:01.245 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:17:01.246 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:17:01.247 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:17:01.248 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 4 Packages 2024-04-26T21:17:01.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:17:01.250 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 82 kB/s | 25 kB 00:00 2024-04-26T21:17:01.250 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 222 M 2024-04-26T21:17:01.250 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-04-26T21:17:01.326 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 225 kB 00:00 2024-04-26T21:17:01.426 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 273 kB/s | 27 kB 00:00 2024-04-26T21:17:01.510 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 385 kB/s | 32 kB 00:00 2024-04-26T21:17:01.685 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.4 MB/s | 11 MB 00:01 2024-04-26T21:17:01.744 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.1 MB/s | 747 kB 00:00 2024-04-26T21:17:01.777 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 163 kB 00:00 2024-04-26T21:17:01.836 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.4 MB/s | 127 kB 00:00 2024-04-26T21:17:01.894 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 4.2 MB/s | 501 kB 00:00 2024-04-26T21:17:01.953 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 785 kB/s | 45 kB 00:00 2024-04-26T21:17:02.028 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T21:17:02.103 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T21:17:02.196 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.5 MB/s | 321 kB 00:00 2024-04-26T21:17:02.312 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T21:17:02.346 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 298 kB 00:00 2024-04-26T21:17:02.388 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T21:17:02.413 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T21:17:02.497 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T21:17:02.572 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 462 kB/s | 34 kB 00:00 2024-04-26T21:17:02.631 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.4 MB/s | 142 kB 00:00 2024-04-26T21:17:02.637 INFO:teuthology.orchestra.run.smithi111.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-26T21:17:02.723 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 9.3 MB/s | 3.1 MB 00:00 2024-04-26T21:17:03.377 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:17:03.379 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:17:03.398 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:17:03.399 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:17:03.400 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:17:03.401 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:17:03.403 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:17:03.406 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:17:03.408 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:17:03.440 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-26T21:17:03.476 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-04-26T21:17:03.495 INFO:teuthology.orchestra.run.smithi111.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:17:03.497 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:17:03.498 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:17:03.499 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:17:03.500 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:17:03.501 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:17:03.502 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:17:03.503 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:17:03.504 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:17:03.505 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:17:03.506 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:17:03.507 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout:Install 148 Packages 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 4 Packages 2024-04-26T21:17:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:17:03.509 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 222 M 2024-04-26T21:17:03.509 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-04-26T21:17:03.523 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 4.0 MB/s | 3.5 MB 00:00 2024-04-26T21:17:03.557 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.1 MB/s | 249 kB 00:00 2024-04-26T21:17:03.582 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 856 kB/s | 49 kB 00:00 2024-04-26T21:17:03.612 INFO:teuthology.orchestra.run.smithi111.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.2 MB/s | 145 kB 00:00 2024-04-26T21:17:03.633 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 224 kB/s | 16 kB 00:00 2024-04-26T21:17:03.666 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 273 kB 00:00 2024-04-26T21:17:03.758 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-26T21:17:03.783 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 157 kB/s | 18 kB 00:00 2024-04-26T21:17:04.421 INFO:teuthology.orchestra.run.smithi111.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-26T21:17:04.479 INFO:teuthology.orchestra.run.smithi111.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.7 MB/s | 2.4 MB 00:00 2024-04-26T21:17:04.609 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 95 kB/s | 80 kB 00:00 2024-04-26T21:17:04.738 INFO:teuthology.orchestra.run.smithi111.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-04-26T21:17:04.815 INFO:teuthology.orchestra.run.smithi143.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-26T21:17:05.434 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 237 kB/s | 195 kB 00:00 2024-04-26T21:17:05.484 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 472 kB/s | 803 kB 00:01 2024-04-26T21:17:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-04-26T21:17:05.840 INFO:teuthology.orchestra.run.smithi143.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 869 kB/s | 889 kB 00:01 2024-04-26T21:17:06.065 INFO:teuthology.orchestra.run.smithi143.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 648 kB/s | 145 kB 00:00 2024-04-26T21:17:06.247 INFO:teuthology.orchestra.run.smithi111.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 5.3 MB/s | 21 MB 00:04 2024-04-26T21:17:06.342 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 279 kB/s | 253 kB 00:00 2024-04-26T21:17:06.347 INFO:teuthology.orchestra.run.smithi111.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 254 kB/s | 25 kB 00:00 2024-04-26T21:17:07.027 INFO:teuthology.orchestra.run.smithi111.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 7.4 MB/s | 17 MB 00:02 2024-04-26T21:17:07.131 INFO:teuthology.orchestra.run.smithi111.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 225 kB 00:00 2024-04-26T21:17:07.206 INFO:teuthology.orchestra.run.smithi111.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 365 kB/s | 27 kB 00:00 2024-04-26T21:17:07.273 INFO:teuthology.orchestra.run.smithi111.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 482 kB/s | 32 kB 00:00 2024-04-26T21:17:07.276 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.1 MB/s | 49 MB 00:06 2024-04-26T21:17:07.432 INFO:teuthology.orchestra.run.smithi111.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T21:17:07.443 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 97 kB/s | 106 kB 00:01 2024-04-26T21:17:07.507 INFO:teuthology.orchestra.run.smithi143.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-26T21:17:07.532 INFO:teuthology.orchestra.run.smithi111.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 163 kB 00:00 2024-04-26T21:17:07.657 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 578 kB/s | 1.2 MB 00:02 2024-04-26T21:17:07.732 INFO:teuthology.orchestra.run.smithi111.stdout:(16/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.0 MB/s | 11 MB 00:02 2024-04-26T21:17:07.766 INFO:teuthology.orchestra.run.smithi111.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 545 kB/s | 127 kB 00:00 2024-04-26T21:17:07.883 INFO:teuthology.orchestra.run.smithi111.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 3.3 MB/s | 501 kB 00:00 2024-04-26T21:17:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 417 kB/s | 45 kB 00:00 2024-04-26T21:17:08.101 INFO:teuthology.orchestra.run.smithi111.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:17:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-26T21:17:08.200 INFO:teuthology.orchestra.run.smithi111.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.6 MB/s | 161 kB 00:00 2024-04-26T21:17:08.249 INFO:teuthology.orchestra.run.smithi143.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-04-26T21:17:08.252 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 139 kB/s | 135 kB 00:00 2024-04-26T21:17:08.309 INFO:teuthology.orchestra.run.smithi111.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 9.9 MB/s | 5.4 MB 00:00 2024-04-26T21:17:08.351 INFO:teuthology.orchestra.run.smithi111.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 321 kB 00:00 2024-04-26T21:17:08.401 INFO:teuthology.orchestra.run.smithi111.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 298 kB 00:00 2024-04-26T21:17:08.411 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 129 kB/s | 125 kB 00:00 2024-04-26T21:17:08.435 INFO:teuthology.orchestra.run.smithi111.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T21:17:08.477 INFO:teuthology.orchestra.run.smithi111.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T21:17:08.560 INFO:teuthology.orchestra.run.smithi111.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T21:17:08.644 INFO:teuthology.orchestra.run.smithi111.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 415 kB/s | 34 kB 00:00 2024-04-26T21:17:08.678 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 211 kB/s | 215 kB 00:01 2024-04-26T21:17:08.753 INFO:teuthology.orchestra.run.smithi111.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.3 MB/s | 142 kB 00:00 2024-04-26T21:17:08.969 INFO:teuthology.orchestra.run.smithi111.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 3.1 MB 00:00 2024-04-26T21:17:09.145 INFO:teuthology.orchestra.run.smithi111.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 9.0 MB/s | 3.5 MB 00:00 2024-04-26T21:17:09.229 INFO:teuthology.orchestra.run.smithi111.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.9 MB/s | 249 kB 00:00 2024-04-26T21:17:09.329 INFO:teuthology.orchestra.run.smithi111.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 495 kB/s | 49 kB 00:00 2024-04-26T21:17:09.413 INFO:teuthology.orchestra.run.smithi111.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 199 kB/s | 16 kB 00:00 2024-04-26T21:17:09.436 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 141 kB/s | 107 kB 00:00 2024-04-26T21:17:09.505 INFO:teuthology.orchestra.run.smithi111.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T21:17:09.621 INFO:teuthology.orchestra.run.smithi111.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T21:17:09.705 INFO:teuthology.orchestra.run.smithi111.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 220 kB/s | 18 kB 00:00 2024-04-26T21:17:09.745 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 462 kB/s | 689 kB 00:01 2024-04-26T21:17:09.822 INFO:teuthology.orchestra.run.smithi111.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 691 kB/s | 80 kB 00:00 2024-04-26T21:17:09.964 INFO:teuthology.orchestra.run.smithi111.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-04-26T21:17:10.023 INFO:teuthology.orchestra.run.smithi111.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-26T21:17:10.045 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 35 kB/s | 21 kB 00:00 2024-04-26T21:17:10.223 INFO:teuthology.orchestra.run.smithi111.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-26T21:17:10.224 INFO:teuthology.orchestra.run.smithi143.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.3 MB/s | 4.7 MB 00:02 2024-04-26T21:17:10.237 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 517 kB/s | 943 kB 00:01 2024-04-26T21:17:10.357 INFO:teuthology.orchestra.run.smithi111.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-26T21:17:10.424 INFO:teuthology.orchestra.run.smithi111.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-26T21:17:10.491 INFO:teuthology.orchestra.run.smithi111.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T21:17:10.516 INFO:teuthology.orchestra.run.smithi111.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 676 kB/s | 106 kB 00:00 2024-04-26T21:17:10.549 INFO:teuthology.orchestra.run.smithi111.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T21:17:10.600 INFO:teuthology.orchestra.run.smithi111.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T21:17:10.720 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-04-26T21:17:10.754 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 311 kB/s | 313 kB 00:01 2024-04-26T21:17:11.075 INFO:teuthology.orchestra.run.smithi111.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 689 kB 00:00 2024-04-26T21:17:11.134 INFO:teuthology.orchestra.run.smithi111.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 943 kB 00:00 2024-04-26T21:17:11.201 INFO:teuthology.orchestra.run.smithi111.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 860 kB/s | 107 kB 00:00 2024-04-26T21:17:11.226 INFO:teuthology.orchestra.run.smithi111.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 236 kB/s | 21 kB 00:00 2024-04-26T21:17:11.271 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-04-26T21:17:11.313 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-04-26T21:17:11.318 INFO:teuthology.orchestra.run.smithi111.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 361 kB/s | 33 kB 00:00 2024-04-26T21:17:11.368 INFO:teuthology.orchestra.run.smithi111.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-26T21:17:11.460 INFO:teuthology.orchestra.run.smithi111.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 201 kB/s | 18 kB 00:00 2024-04-26T21:17:11.552 INFO:teuthology.orchestra.run.smithi111.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 11 kB 00:00 2024-04-26T21:17:11.866 INFO:teuthology.orchestra.run.smithi143.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 2.9 MB/s | 21 MB 00:07 2024-04-26T21:17:11.933 INFO:teuthology.orchestra.run.smithi143.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 384 kB/s | 25 kB 00:00 2024-04-26T21:17:11.963 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-04-26T21:17:12.055 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 205 kB/s | 160 kB 00:00 2024-04-26T21:17:12.086 INFO:teuthology.orchestra.run.smithi111.stdout:(56/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.6 MB/s | 49 MB 00:05 2024-04-26T21:17:12.119 INFO:teuthology.orchestra.run.smithi111.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 283 kB/s | 160 kB 00:00 2024-04-26T21:17:12.200 INFO:teuthology.orchestra.run.smithi143.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 4.3 MB/s | 17 MB 00:03 2024-04-26T21:17:12.237 INFO:teuthology.orchestra.run.smithi111.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-04-26T21:17:12.275 INFO:teuthology.orchestra.run.smithi143.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.0 MB/s | 225 kB 00:00 2024-04-26T21:17:12.325 INFO:teuthology.orchestra.run.smithi143.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 550 kB/s | 27 kB 00:00 2024-04-26T21:17:12.337 INFO:teuthology.orchestra.run.smithi111.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 181 kB/s | 45 kB 00:00 2024-04-26T21:17:12.376 INFO:teuthology.orchestra.run.smithi143.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 646 kB/s | 32 kB 00:00 2024-04-26T21:17:12.412 INFO:teuthology.orchestra.run.smithi111.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 352 kB/s | 25 kB 00:00 2024-04-26T21:17:12.479 INFO:teuthology.orchestra.run.smithi111.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 721 kB/s | 47 kB 00:00 2024-04-26T21:17:12.518 INFO:teuthology.orchestra.run.smithi143.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 5.2 MB/s | 747 kB 00:00 2024-04-26T21:17:12.521 INFO:teuthology.orchestra.run.smithi111.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-26T21:17:12.571 INFO:teuthology.orchestra.run.smithi111.stdout:(63/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.4 MB/s | 662 kB 00:00 2024-04-26T21:17:12.576 INFO:teuthology.orchestra.run.smithi143.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.8 MB/s | 163 kB 00:00 2024-04-26T21:17:12.627 INFO:teuthology.orchestra.run.smithi143.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:17:12.638 INFO:teuthology.orchestra.run.smithi111.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 536 kB/s | 35 kB 00:00 2024-04-26T21:17:12.671 INFO:teuthology.orchestra.run.smithi111.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-04-26T21:17:12.697 INFO:teuthology.orchestra.run.smithi111.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-26T21:17:12.745 INFO:teuthology.orchestra.run.smithi143.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 4.2 MB/s | 501 kB 00:00 2024-04-26T21:17:12.839 INFO:teuthology.orchestra.run.smithi111.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T21:17:12.948 INFO:teuthology.orchestra.run.smithi111.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.3 MB/s | 911 kB 00:00 2024-04-26T21:17:13.023 INFO:teuthology.orchestra.run.smithi111.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-04-26T21:17:13.248 INFO:teuthology.orchestra.run.smithi111.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-04-26T21:17:13.398 INFO:teuthology.orchestra.run.smithi111.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-26T21:17:13.430 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 424 kB/s | 582 kB 00:01 2024-04-26T21:17:13.435 INFO:teuthology.orchestra.run.smithi143.stdout:(18/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 3.3 MB/s | 11 MB 00:03 2024-04-26T21:17:13.489 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 435 kB/s | 662 kB 00:01 2024-04-26T21:17:13.507 INFO:teuthology.orchestra.run.smithi111.stdout:(72/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.5 MB/s | 4.7 MB 00:00 2024-04-26T21:17:13.540 INFO:teuthology.orchestra.run.smithi111.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 620 kB/s | 87 kB 00:00 2024-04-26T21:17:13.582 INFO:teuthology.orchestra.run.smithi111.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-04-26T21:17:13.585 INFO:teuthology.orchestra.run.smithi143.stdout:(19/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 6.4 MB/s | 5.4 MB 00:00 2024-04-26T21:17:13.611 INFO:teuthology.orchestra.run.smithi143.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 258 kB/s | 45 kB 00:00 2024-04-26T21:17:13.641 INFO:teuthology.orchestra.run.smithi111.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-26T21:17:13.645 INFO:teuthology.orchestra.run.smithi143.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 2.3 MB/s | 135 kB 00:00 2024-04-26T21:17:13.683 INFO:teuthology.orchestra.run.smithi111.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-26T21:17:13.687 INFO:teuthology.orchestra.run.smithi143.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T21:17:13.733 INFO:teuthology.orchestra.run.smithi111.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 282 kB/s | 26 kB 00:00 2024-04-26T21:17:13.737 INFO:teuthology.orchestra.run.smithi143.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.4 MB/s | 321 kB 00:00 2024-04-26T21:17:13.758 INFO:teuthology.orchestra.run.smithi111.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 257 kB/s | 19 kB 00:00 2024-04-26T21:17:13.787 INFO:teuthology.orchestra.run.smithi143.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 99 kB 00:00 2024-04-26T21:17:13.821 INFO:teuthology.orchestra.run.smithi143.stdout:(25/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.2 MB/s | 298 kB 00:00 2024-04-26T21:17:13.825 INFO:teuthology.orchestra.run.smithi111.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T21:17:13.847 INFO:teuthology.orchestra.run.smithi143.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T21:17:13.884 INFO:teuthology.orchestra.run.smithi111.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 279 kB/s | 35 kB 00:00 2024-04-26T21:17:13.913 INFO:teuthology.orchestra.run.smithi143.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T21:17:13.955 INFO:teuthology.orchestra.run.smithi143.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 842 kB/s | 34 kB 00:00 2024-04-26T21:17:14.006 INFO:teuthology.orchestra.run.smithi143.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T21:17:14.043 INFO:teuthology.orchestra.run.smithi111.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-26T21:17:14.114 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 37 kB/s | 25 kB 00:00 2024-04-26T21:17:14.159 INFO:teuthology.orchestra.run.smithi111.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-04-26T21:17:14.214 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-04-26T21:17:14.226 INFO:teuthology.orchestra.run.smithi111.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-26T21:17:14.252 INFO:teuthology.orchestra.run.smithi111.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 845 kB/s | 77 kB 00:00 2024-04-26T21:17:14.377 INFO:teuthology.orchestra.run.smithi111.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.3 MB/s | 5.1 MB 00:00 2024-04-26T21:17:14.402 INFO:teuthology.orchestra.run.smithi111.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 241 kB/s | 42 kB 00:00 2024-04-26T21:17:14.461 INFO:teuthology.orchestra.run.smithi111.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-04-26T21:17:14.464 INFO:teuthology.orchestra.run.smithi143.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.7 MB/s | 3.5 MB 00:00 2024-04-26T21:17:14.528 INFO:teuthology.orchestra.run.smithi111.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-26T21:17:14.562 INFO:teuthology.orchestra.run.smithi111.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-26T21:17:14.595 INFO:teuthology.orchestra.run.smithi111.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-04-26T21:17:14.629 INFO:teuthology.orchestra.run.smithi111.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-26T21:17:14.696 INFO:teuthology.orchestra.run.smithi111.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-04-26T21:17:14.706 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 685 kB/s | 3.0 MB 00:04 2024-04-26T21:17:14.738 INFO:teuthology.orchestra.run.smithi111.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-04-26T21:17:14.838 INFO:teuthology.orchestra.run.smithi111.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-26T21:17:14.880 INFO:teuthology.orchestra.run.smithi111.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 254 kB/s | 36 kB 00:00 2024-04-26T21:17:14.972 INFO:teuthology.orchestra.run.smithi111.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 357 kB/s | 47 kB 00:00 2024-04-26T21:17:15.022 INFO:teuthology.orchestra.run.smithi111.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-04-26T21:17:15.047 INFO:teuthology.orchestra.run.smithi111.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 778 kB/s | 58 kB 00:00 2024-04-26T21:17:15.106 INFO:teuthology.orchestra.run.smithi111.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T21:17:15.131 INFO:teuthology.orchestra.run.smithi111.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 770 kB/s | 64 kB 00:00 2024-04-26T21:17:15.156 INFO:teuthology.orchestra.run.smithi143.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 2.3 MB/s | 3.1 MB 00:01 2024-04-26T21:17:15.181 INFO:teuthology.orchestra.run.smithi111.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-26T21:17:15.306 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 173 kB/s | 188 kB 00:01 2024-04-26T21:17:15.340 INFO:teuthology.orchestra.run.smithi111.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-04-26T21:17:15.348 INFO:teuthology.orchestra.run.smithi143.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-04-26T21:17:15.349 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 237 kB/s | 292 kB 00:01 2024-04-26T21:17:15.382 INFO:teuthology.orchestra.run.smithi143.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.1 MB/s | 249 kB 00:00 2024-04-26T21:17:15.398 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 51 kB/s | 35 kB 00:00 2024-04-26T21:17:15.407 INFO:teuthology.orchestra.run.smithi143.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 857 kB/s | 49 kB 00:00 2024-04-26T21:17:15.458 INFO:teuthology.orchestra.run.smithi143.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 221 kB/s | 16 kB 00:00 2024-04-26T21:17:15.483 INFO:teuthology.orchestra.run.smithi111.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-04-26T21:17:15.508 INFO:teuthology.orchestra.run.smithi143.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 273 kB 00:00 2024-04-26T21:17:15.558 INFO:teuthology.orchestra.run.smithi143.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 372 kB/s | 18 kB 00:00 2024-04-26T21:17:15.725 INFO:teuthology.orchestra.run.smithi143.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 485 kB/s | 80 kB 00:00 2024-04-26T21:17:15.827 INFO:teuthology.orchestra.run.smithi143.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.0 MB/s | 762 kB 00:00 2024-04-26T21:17:15.833 INFO:teuthology.orchestra.run.smithi111.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T21:17:15.908 INFO:teuthology.orchestra.run.smithi111.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 263 kB/s | 19 kB 00:00 2024-04-26T21:17:15.942 INFO:teuthology.orchestra.run.smithi111.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 674 kB/s | 309 kB 00:00 2024-04-26T21:17:16.017 INFO:teuthology.orchestra.run.smithi111.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-04-26T21:17:16.101 INFO:teuthology.orchestra.run.smithi111.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 588 kB/s | 49 kB 00:00 2024-04-26T21:17:16.151 INFO:teuthology.orchestra.run.smithi143.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-26T21:17:16.234 INFO:teuthology.orchestra.run.smithi143.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 481 kB/s | 195 kB 00:00 2024-04-26T21:17:16.276 INFO:teuthology.orchestra.run.smithi111.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-26T21:17:16.301 INFO:teuthology.orchestra.run.smithi111.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 338 kB/s | 67 kB 00:00 2024-04-26T21:17:16.343 INFO:teuthology.orchestra.run.smithi143.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-26T21:17:16.360 INFO:teuthology.orchestra.run.smithi111.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-26T21:17:16.427 INFO:teuthology.orchestra.run.smithi143.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-26T21:17:16.494 INFO:teuthology.orchestra.run.smithi143.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T21:17:16.569 INFO:teuthology.orchestra.run.smithi143.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-26T21:17:16.577 INFO:teuthology.orchestra.run.smithi111.stdout:(112/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.1 MB/s | 16 MB 00:02 2024-04-26T21:17:16.627 INFO:teuthology.orchestra.run.smithi111.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-04-26T21:17:16.678 INFO:teuthology.orchestra.run.smithi143.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T21:17:16.735 INFO:teuthology.orchestra.run.smithi111.stdout:(114/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-04-26T21:17:16.769 INFO:teuthology.orchestra.run.smithi111.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 438 kB/s | 32 kB 00:00 2024-04-26T21:17:16.795 INFO:teuthology.orchestra.run.smithi143.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-04-26T21:17:16.828 INFO:teuthology.orchestra.run.smithi111.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-26T21:17:16.862 INFO:teuthology.orchestra.run.smithi111.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.9 MB/s | 172 kB 00:00 2024-04-26T21:17:16.920 INFO:teuthology.orchestra.run.smithi111.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-26T21:17:16.946 INFO:teuthology.orchestra.run.smithi111.stdout:(119/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 118 kB/s | 43 kB 00:00 2024-04-26T21:17:16.979 INFO:teuthology.orchestra.run.smithi111.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-26T21:17:17.000 INFO:teuthology.orchestra.run.smithi143.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-26T21:17:17.004 INFO:teuthology.orchestra.run.smithi111.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 424 kB/s | 35 kB 00:00 2024-04-26T21:17:17.062 INFO:teuthology.orchestra.run.smithi143.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 107 kB 00:00 2024-04-26T21:17:17.088 INFO:teuthology.orchestra.run.smithi111.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-26T21:17:17.129 INFO:teuthology.orchestra.run.smithi143.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-26T21:17:17.146 INFO:teuthology.orchestra.run.smithi111.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T21:17:17.154 INFO:teuthology.orchestra.run.smithi143.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 236 kB/s | 21 kB 00:00 2024-04-26T21:17:17.172 INFO:teuthology.orchestra.run.smithi111.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-26T21:17:17.188 INFO:teuthology.orchestra.run.smithi143.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-26T21:17:17.197 INFO:teuthology.orchestra.run.smithi111.stdout:(125/152): python3-google-auth-2.29.0-1.el9.noa 820 kB/s | 206 kB 00:00 2024-04-26T21:17:17.213 INFO:teuthology.orchestra.run.smithi143.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 564 kB/s | 33 kB 00:00 2024-04-26T21:17:17.222 INFO:teuthology.orchestra.run.smithi111.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-26T21:17:17.248 INFO:teuthology.orchestra.run.smithi111.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-04-26T21:17:17.272 INFO:teuthology.orchestra.run.smithi143.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-26T21:17:17.273 INFO:teuthology.orchestra.run.smithi111.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-26T21:17:17.298 INFO:teuthology.orchestra.run.smithi111.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-04-26T21:17:17.331 INFO:teuthology.orchestra.run.smithi143.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 194 kB/s | 11 kB 00:00 2024-04-26T21:17:17.357 INFO:teuthology.orchestra.run.smithi111.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 559 kB/s | 46 kB 00:00 2024-04-26T21:17:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 731 kB/s | 79 kB 00:00 2024-04-26T21:17:17.414 INFO:teuthology.orchestra.run.smithi143.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-26T21:17:17.449 INFO:teuthology.orchestra.run.smithi111.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-26T21:17:17.473 INFO:teuthology.orchestra.run.smithi143.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 787 kB/s | 45 kB 00:00 2024-04-26T21:17:17.474 INFO:teuthology.orchestra.run.smithi111.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-04-26T21:17:17.508 INFO:teuthology.orchestra.run.smithi111.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T21:17:17.533 INFO:teuthology.orchestra.run.smithi111.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-26T21:17:17.548 INFO:teuthology.orchestra.run.smithi143.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.3 MB/s | 3.0 MB 00:00 2024-04-26T21:17:17.558 INFO:teuthology.orchestra.run.smithi111.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T21:17:17.584 INFO:teuthology.orchestra.run.smithi111.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-04-26T21:17:17.598 INFO:teuthology.orchestra.run.smithi143.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-26T21:17:17.609 INFO:teuthology.orchestra.run.smithi111.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-04-26T21:17:17.649 INFO:teuthology.orchestra.run.smithi143.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-26T21:17:17.659 INFO:teuthology.orchestra.run.smithi111.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-26T21:17:17.674 INFO:teuthology.orchestra.run.smithi143.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-26T21:17:17.684 INFO:teuthology.orchestra.run.smithi111.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 594 kB/s | 59 kB 00:00 2024-04-26T21:17:17.708 INFO:teuthology.orchestra.run.smithi143.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 821 kB/s | 47 kB 00:00 2024-04-26T21:17:17.710 INFO:teuthology.orchestra.run.smithi111.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-04-26T21:17:17.741 INFO:teuthology.orchestra.run.smithi143.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T21:17:17.760 INFO:teuthology.orchestra.run.smithi111.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 688 kB/s | 51 kB 00:00 2024-04-26T21:17:17.785 INFO:teuthology.orchestra.run.smithi111.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T21:17:17.800 INFO:teuthology.orchestra.run.smithi143.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T21:17:17.819 INFO:teuthology.orchestra.run.smithi111.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-26T21:17:17.825 INFO:teuthology.orchestra.run.smithi143.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 425 kB/s | 35 kB 00:00 2024-04-26T21:17:17.861 INFO:teuthology.orchestra.run.smithi111.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-26T21:17:17.886 INFO:teuthology.orchestra.run.smithi111.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-26T21:17:17.920 INFO:teuthology.orchestra.run.smithi111.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-26T21:17:17.995 INFO:teuthology.orchestra.run.smithi111.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T21:17:18.149 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 592 kB/s | 1.6 MB 00:02 2024-04-26T21:17:18.334 INFO:teuthology.orchestra.run.smithi143.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-04-26T21:17:18.426 INFO:teuthology.orchestra.run.smithi143.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.6 MB/s | 4.7 MB 00:00 2024-04-26T21:17:18.445 INFO:teuthology.orchestra.run.smithi111.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-04-26T21:17:18.488 INFO:teuthology.orchestra.run.smithi111.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-04-26T21:17:18.577 INFO:teuthology.orchestra.run.smithi143.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-04-26T21:17:18.579 INFO:teuthology.orchestra.run.smithi111.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.9 MB/s | 3.4 MB 00:00 2024-04-26T21:17:18.627 INFO:teuthology.orchestra.run.smithi143.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-26T21:17:18.660 INFO:teuthology.orchestra.run.smithi143.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T21:17:18.854 INFO:teuthology.orchestra.run.smithi111.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.9 MB/s | 2.4 MB 00:00 2024-04-26T21:17:18.861 INFO:teuthology.orchestra.run.smithi143.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-04-26T21:17:18.870 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:17:18.871 INFO:teuthology.orchestra.run.smithi111.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T21:17:18.943 INFO:teuthology.orchestra.run.smithi143.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-26T21:17:19.295 INFO:teuthology.orchestra.run.smithi143.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 8.7 MB/s | 5.8 MB 00:00 2024-04-26T21:17:19.370 INFO:teuthology.orchestra.run.smithi143.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T21:17:19.437 INFO:teuthology.orchestra.run.smithi143.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-26T21:17:19.496 INFO:teuthology.orchestra.run.smithi143.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 447 kB/s | 26 kB 00:00 2024-04-26T21:17:19.613 INFO:teuthology.orchestra.run.smithi143.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.9 MB/s | 6.0 MB 00:00 2024-04-26T21:17:19.638 INFO:teuthology.orchestra.run.smithi143.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 136 kB/s | 19 kB 00:00 2024-04-26T21:17:19.671 INFO:teuthology.orchestra.run.smithi143.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:17:19.721 INFO:teuthology.orchestra.run.smithi143.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 423 kB/s | 35 kB 00:00 2024-04-26T21:17:19.798 INFO:teuthology.orchestra.run.smithi143.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T21:17:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 547 kB/s | 911 kB 00:01 2024-04-26T21:17:19.864 INFO:teuthology.orchestra.run.smithi143.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T21:17:19.923 INFO:teuthology.orchestra.run.smithi143.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:17:19.982 INFO:teuthology.orchestra.run.smithi143.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 737 kB/s | 42 kB 00:00 2024-04-26T21:17:20.057 INFO:teuthology.orchestra.run.smithi143.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T21:17:20.141 INFO:teuthology.orchestra.run.smithi143.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T21:17:20.241 INFO:teuthology.orchestra.run.smithi143.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.0 MB/s | 5.1 MB 00:00 2024-04-26T21:17:20.275 INFO:teuthology.orchestra.run.smithi143.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 279 kB 00:00 2024-04-26T21:17:20.308 INFO:teuthology.orchestra.run.smithi143.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-04-26T21:17:20.342 INFO:teuthology.orchestra.run.smithi143.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 809 kB/s | 54 kB 00:00 2024-04-26T21:17:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 559 kB/s | 42 kB 00:00 2024-04-26T21:17:20.509 INFO:teuthology.orchestra.run.smithi143.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 486 kB/s | 44 kB 00:00 2024-04-26T21:17:20.593 INFO:teuthology.orchestra.run.smithi143.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-26T21:17:20.688 INFO:teuthology.orchestra.run.smithi143.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 384 kB/s | 36 kB 00:00 2024-04-26T21:17:20.766 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 140 kB/s | 133 kB 00:00 2024-04-26T21:17:20.788 INFO:teuthology.orchestra.run.smithi143.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 478 kB/s | 47 kB 00:00 2024-04-26T21:17:20.897 INFO:teuthology.orchestra.run.smithi143.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-26T21:17:20.988 INFO:teuthology.orchestra.run.smithi143.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 636 kB/s | 58 kB 00:00 2024-04-26T21:17:21.072 INFO:teuthology.orchestra.run.smithi143.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T21:17:21.164 INFO:teuthology.orchestra.run.smithi143.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 700 kB/s | 64 kB 00:00 2024-04-26T21:17:21.265 INFO:teuthology.orchestra.run.smithi143.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-26T21:17:21.673 INFO:teuthology.orchestra.run.smithi143.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2024-04-26T21:17:21.676 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-04-26T21:17:21.920 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-04-26T21:17:21.920 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-04-26T21:17:22.067 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 674 kB/s | 4.4 MB 00:06 2024-04-26T21:17:22.490 INFO:teuthology.orchestra.run.smithi143.stdout:(102/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 4.7 MB/s | 49 MB 00:10 2024-04-26T21:17:22.591 INFO:teuthology.orchestra.run.smithi143.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 942 kB/s | 863 kB 00:00 2024-04-26T21:17:22.616 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 664 kB/s | 4.7 MB 00:07 2024-04-26T21:17:23.183 INFO:teuthology.orchestra.run.smithi143.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 523 kB/s | 309 kB 00:00 2024-04-26T21:17:23.274 INFO:teuthology.orchestra.run.smithi143.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 214 kB/s | 19 kB 00:00 2024-04-26T21:17:23.375 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 116 kB/s | 87 kB 00:00 2024-04-26T21:17:23.408 INFO:teuthology.orchestra.run.smithi143.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 598 kB/s | 548 kB 00:00 2024-04-26T21:17:23.500 INFO:teuthology.orchestra.run.smithi143.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-04-26T21:17:23.592 INFO:teuthology.orchestra.run.smithi143.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 534 kB/s | 49 kB 00:00 2024-04-26T21:17:23.709 INFO:teuthology.orchestra.run.smithi143.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T21:17:23.734 INFO:teuthology.orchestra.run.smithi143.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-04-26T21:17:23.801 INFO:teuthology.orchestra.run.smithi143.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-26T21:17:23.918 INFO:teuthology.orchestra.run.smithi143.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-26T21:17:24.052 INFO:teuthology.orchestra.run.smithi143.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2024-04-26T21:17:24.083 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 513 kB/s | 1.0 MB 00:02 2024-04-26T21:17:24.144 INFO:teuthology.orchestra.run.smithi143.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 354 kB/s | 32 kB 00:00 2024-04-26T21:17:24.236 INFO:teuthology.orchestra.run.smithi143.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 165 kB/s | 15 kB 00:00 2024-04-26T21:17:24.328 INFO:teuthology.orchestra.run.smithi143.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:17:24.428 INFO:teuthology.orchestra.run.smithi143.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T21:17:24.471 INFO:teuthology.orchestra.run.smithi143.stdout:(118/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-04-26T21:17:24.550 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-04-26T21:17:24.551 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-04-26T21:17:24.562 INFO:teuthology.orchestra.run.smithi143.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 388 kB/s | 35 kB 00:00 2024-04-26T21:17:24.663 INFO:teuthology.orchestra.run.smithi143.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-26T21:17:24.696 INFO:teuthology.orchestra.run.smithi143.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-04-26T21:17:24.847 INFO:teuthology.orchestra.run.smithi143.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 968 kB/s | 144 kB 00:00 2024-04-26T21:17:24.931 INFO:teuthology.orchestra.run.smithi143.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-26T21:17:24.975 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 274 kB/s | 244 kB 00:00 2024-04-26T21:17:25.014 INFO:teuthology.orchestra.run.smithi143.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 213 kB/s | 18 kB 00:00 2024-04-26T21:17:25.098 INFO:teuthology.orchestra.run.smithi143.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-26T21:17:25.198 INFO:teuthology.orchestra.run.smithi143.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-04-26T21:17:25.432 INFO:teuthology.orchestra.run.smithi143.stdout:(127/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-04-26T21:17:25.457 INFO:teuthology.orchestra.run.smithi143.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 76 kB/s | 20 kB 00:00 2024-04-26T21:17:25.516 INFO:teuthology.orchestra.run.smithi143.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-26T21:17:25.574 INFO:teuthology.orchestra.run.smithi143.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-04-26T21:17:25.608 INFO:teuthology.orchestra.run.smithi143.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 863 kB/s | 79 kB 00:00 2024-04-26T21:17:25.675 INFO:teuthology.orchestra.run.smithi143.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-04-26T21:17:25.709 INFO:teuthology.orchestra.run.smithi143.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T21:17:25.759 INFO:teuthology.orchestra.run.smithi143.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-04-26T21:17:25.801 INFO:teuthology.orchestra.run.smithi143.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 984 kB/s | 90 kB 00:00 2024-04-26T21:17:25.825 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 292 kB/s | 248 kB 00:00 2024-04-26T21:17:25.851 INFO:teuthology.orchestra.run.smithi143.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 309 kB/s | 28 kB 00:00 2024-04-26T21:17:25.893 INFO:teuthology.orchestra.run.smithi143.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 337 kB/s | 31 kB 00:00 2024-04-26T21:17:25.993 INFO:teuthology.orchestra.run.smithi143.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 594 kB/s | 59 kB 00:00 2024-04-26T21:17:26.018 INFO:teuthology.orchestra.run.smithi143.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-26T21:17:26.061 INFO:teuthology.orchestra.run.smithi143.stdout:(140/152): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-04-26T21:17:26.086 INFO:teuthology.orchestra.run.smithi143.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-04-26T21:17:26.111 INFO:teuthology.orchestra.run.smithi143.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-26T21:17:26.153 INFO:teuthology.orchestra.run.smithi143.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-26T21:17:26.170 INFO:teuthology.orchestra.run.smithi143.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-26T21:17:26.246 INFO:teuthology.orchestra.run.smithi143.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-26T21:17:26.279 INFO:teuthology.orchestra.run.smithi143.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T21:17:26.346 INFO:teuthology.orchestra.run.smithi143.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2024-04-26T21:17:26.392 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-04-26T21:17:26.422 INFO:teuthology.orchestra.run.smithi143.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-26T21:17:26.789 INFO:teuthology.orchestra.run.smithi143.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.7 MB/s | 3.4 MB 00:00 2024-04-26T21:17:26.872 INFO:teuthology.orchestra.run.smithi143.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 1.3 MB 00:00 2024-04-26T21:17:26.934 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-04-26T21:17:27.422 INFO:teuthology.orchestra.run.smithi143.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 2.4 MB 00:00 2024-04-26T21:17:27.598 INFO:teuthology.orchestra.run.smithi143.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-26T21:17:27.606 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:17:27.607 INFO:teuthology.orchestra.run.smithi143.stdout:Total 9.2 MB/s | 222 MB 00:24 2024-04-26T21:17:27.726 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 218 kB/s | 172 kB 00:00 2024-04-26T21:17:28.359 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-04-26T21:17:28.626 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 753 kB/s | 5.8 MB 00:07 2024-04-26T21:17:29.551 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 288 kB/s | 250 kB 00:00 2024-04-26T21:17:30.385 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 267 kB/s | 222 kB 00:00 2024-04-26T21:17:30.460 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-04-26T21:17:30.665 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-04-26T21:17:30.665 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-04-26T21:17:30.873 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-04-26T21:17:30.911 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 810 kB/s | 6.0 MB 00:07 2024-04-26T21:17:30.977 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:17:31.085 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-04-26T21:17:31.119 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:17:31.464 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:17:31.538 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:17:31.572 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:17:31.587 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-04-26T21:17:31.713 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:17:32.061 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 277 kB/s | 269 kB 00:00 2024-04-26T21:17:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:17:32.261 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:17:32.397 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 196 kB/s | 159 kB 00:00 2024-04-26T21:17:32.427 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:17:32.461 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:17:32.595 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:17:32.631 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:17:32.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:17:32.980 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 88 kB/s | 51 kB 00:00 2024-04-26T21:17:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 291 kB/s | 279 kB 00:00 2024-04-26T21:17:33.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:17:33.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:17:33.243 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:17:33.275 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:17:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:17:33.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:17:33.461 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-04-26T21:17:33.462 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-04-26T21:17:33.471 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:17:33.614 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-04-26T21:17:33.800 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:17:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:17:34.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:17:34.194 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:17:34.231 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-04-26T21:17:34.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:17:34.295 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:17:34.400 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:17:34.449 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:17:34.520 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:17:34.572 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:17:34.619 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:17:34.752 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:17:34.850 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:17:34.888 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:17:34.946 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:17:34.948 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 792 kB/s | 5.1 MB 00:06 2024-04-26T21:17:34.973 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 60 kB/s | 44 kB 00:00 2024-04-26T21:17:35.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:17:35.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:17:35.132 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:17:35.163 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:17:35.276 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:17:35.333 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:17:35.468 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:17:35.624 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 55 kB/s | 36 kB 00:00 2024-04-26T21:17:35.650 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:17:35.721 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:17:35.765 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 271 kB/s | 221 kB 00:00 2024-04-26T21:17:36.291 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-04-26T21:17:36.432 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:17:36.543 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:17:36.587 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:17:36.640 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:17:36.683 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:17:36.747 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:17:36.790 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 298 kB/s | 305 kB 00:01 2024-04-26T21:17:36.966 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-04-26T21:17:36.992 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:17:37.049 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:17:37.151 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:17:37.257 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:17:37.299 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:17:37.338 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:17:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:17:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:17:37.507 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:17:37.565 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:17:37.625 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 112 kB/s | 94 kB 00:00 2024-04-26T21:17:37.692 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 88 kB/s | 64 kB 00:00 2024-04-26T21:17:37.742 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 459 kB/s | 22 kB 00:00 2024-04-26T21:17:37.763 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:17:37.819 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:17:37.850 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-26T21:17:37.864 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:17:37.917 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:17:38.051 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:17:38.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:17:38.137 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:17:38.180 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:17:38.213 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:17:38.276 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-04-26T21:17:38.401 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T21:17:38.417 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:17:38.493 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 213 kB/s | 19 kB 00:00 2024-04-26T21:17:38.600 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:17:38.644 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:17:38.669 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 788 kB/s | 309 kB 00:00 2024-04-26T21:17:38.683 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:17:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:17:38.760 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-04-26T21:17:38.787 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:17:38.861 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-26T21:17:38.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:17:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-04-26T21:17:38.911 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:17:38.945 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 815 kB/s | 67 kB 00:00 2024-04-26T21:17:38.962 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:17:38.978 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-26T21:17:39.010 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:17:39.053 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-04-26T21:17:39.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:17:39.129 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 587 kB/s | 43 kB 00:00 2024-04-26T21:17:39.133 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:17:39.171 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-26T21:17:39.213 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 388 kB/s | 32 kB 00:00 2024-04-26T21:17:39.255 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-04-26T21:17:39.274 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:17:39.305 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:17:39.340 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:17:39.355 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T21:17:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:17:39.397 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T21:17:39.439 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 425 kB/s | 35 kB 00:00 2024-04-26T21:17:39.462 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:17:39.481 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T21:17:39.573 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-26T21:17:39.617 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:17:39.648 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-26T21:17:39.674 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T21:17:39.696 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:17:39.732 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-26T21:17:39.758 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 279 kB/s | 23 kB 00:00 2024-04-26T21:17:39.816 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-26T21:17:39.842 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-26T21:17:39.933 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-26T21:17:39.959 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-04-26T21:17:39.973 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-04-26T21:17:40.017 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 952 kB/s | 79 kB 00:00 2024-04-26T21:17:40.043 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 695 kB/s | 58 kB 00:00 2024-04-26T21:17:40.105 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:17:40.109 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T21:17:40.135 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-26T21:17:40.185 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:17:40.193 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T21:17:40.219 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-04-26T21:17:40.291 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:17:40.296 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 304 kB/s | 31 kB 00:00 2024-04-26T21:17:40.327 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T21:17:40.350 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:17:40.378 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 732 kB/s | 59 kB 00:00 2024-04-26T21:17:40.389 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:17:40.411 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-04-26T21:17:40.462 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 615 kB/s | 51 kB 00:00 2024-04-26T21:17:40.503 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-26T21:17:40.520 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:17:40.545 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-26T21:17:40.596 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-26T21:17:40.630 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-04-26T21:17:40.666 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:17:40.688 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-04-26T21:17:40.728 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:17:40.763 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:17:40.888 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:17:40.940 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:17:40.965 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:17:41.065 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:17:41.067 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:17:41.105 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:17:41.114 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): librados2-19.0.0-3306.gbc612ffb.el9. 8.0 MB/s | 3.4 MB 00:00 2024-04-26T21:17:41.256 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:17:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-04-26T21:17:41.281 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:17:41.351 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:17:41.377 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:17:41.389 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:17:41.464 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-26T21:17:41.465 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:17:41.475 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:17:41.810 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:17:41.886 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:17:41.911 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:17:42.063 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:17:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:17:42.199 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:17:42.246 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:17:42.311 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:17:42.327 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:17:42.458 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:17:42.799 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:17:42.855 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:17:43.052 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:17:43.173 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:17:43.222 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:17:43.270 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:17:43.373 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:17:43.406 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 646 kB/s | 1.3 MB 00:02 2024-04-26T21:17:43.426 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:17:43.488 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:17:43.530 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:17:43.570 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:17:43.687 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:17:43.757 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:17:43.798 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:17:43.847 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:17:43.949 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:17:43.993 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:17:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:17:44.077 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:17:44.083 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:17:44.167 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:17:44.201 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:17:44.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:17:44.262 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:17:44.270 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:17:44.359 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:17:44.528 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:17:44.597 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:17:44.631 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 774 kB/s | 2.4 MB 00:03 2024-04-26T21:17:44.670 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:17:44.721 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:17:44.843 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:17:44.886 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:17:44.942 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:17:45.267 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:17:45.377 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:17:45.423 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:17:45.469 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:17:45.482 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:17:45.511 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:17:45.516 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:17:45.570 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:17:45.607 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:17:45.634 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:17:45.781 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:17:45.859 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:17:45.917 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:17:46.011 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:17:46.086 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:17:46.133 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:17:46.205 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:17:46.250 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:17:46.315 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:17:46.380 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:17:46.582 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:17:46.631 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:17:46.669 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:17:46.727 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:17:46.865 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:17:46.918 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:17:46.969 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:17:47.009 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:17:47.042 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:17:47.225 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:17:47.408 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:17:47.457 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:17:47.502 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:17:47.553 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:17:47.621 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:17:47.689 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:17:47.738 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:17:47.793 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:17:47.843 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:17:47.917 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:17:47.963 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:17:48.126 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:17:48.195 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:17:48.238 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:17:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:17:48.332 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:48.373 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:48.455 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:17:48.526 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:17:49.023 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:49.068 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:17:49.226 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:17:49.226 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:17:49.227 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:17:49.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:17:49.269 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:17:49.472 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:17:49.534 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:17:49.584 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:17:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:17:49.779 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:17:49.877 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:17:49.920 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:17:50.170 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:17:50.200 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:17:50.305 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:17:51.149 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:17:51.288 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:17:52.189 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 868 kB/s | 16 MB 00:19 2024-04-26T21:17:52.197 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:17:52.198 INFO:teuthology.orchestra.run.smithi097.stdout:Total 4.0 MB/s | 222 MB 00:55 2024-04-26T21:17:52.861 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:17:52.958 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:17:53.002 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:17:53.060 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:17:53.452 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:17:53.496 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:17:53.608 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:17:53.644 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:17:53.699 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:17:54.234 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:17:54.266 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:17:54.358 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:17:54.385 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:17:55.133 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-26T21:17:55.374 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-26T21:17:55.374 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-26T21:17:57.055 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:57.090 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:57.734 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:17:57.786 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:17:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:17:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:17:57.944 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:17:57.986 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:17:58.086 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-26T21:17:58.087 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-04-26T21:18:02.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:02.650 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:02.698 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:02.698 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:02.698 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:18:02.698 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:04.343 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:04.387 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:04.429 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:04.495 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:18:04.564 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:18:04.662 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:18:04.685 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:04.712 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:18:04.746 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:18:04.756 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:04.757 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:04.757 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:18:04.757 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:04.757 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:04.757 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:04.870 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:18:05.244 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:05.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:18:05.322 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:05.322 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:05.322 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:18:05.322 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:05.322 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:05.323 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:05.411 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:18:05.462 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:18:05.526 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:18:05.585 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:18:05.601 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:18:05.620 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:18:05.756 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:18:05.802 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:18:05.835 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:18:05.890 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:18:05.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:18:06.094 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:18:06.182 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:18:06.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:18:06.263 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:18:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:18:06.338 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:18:06.424 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:18:06.456 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:18:06.456 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:18:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:18:06.579 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:18:06.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:18:06.670 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:18:06.999 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:18:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:18:07.147 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:07.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:07.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:18:07.258 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:18:07.360 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:18:07.378 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:18:07.402 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:07.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:18:07.474 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:18:07.576 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:18:07.630 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:18:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:18:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:18:07.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:18:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:18:07.972 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:18:08.175 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:18:08.213 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:18:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:18:08.295 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:18:08.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:18:08.466 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:18:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:18:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:08.747 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:18:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:08.809 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:18:09.445 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:18:09.552 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:18:09.585 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:18:09.629 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:18:09.669 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:18:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:18:09.884 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:09.913 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:09.941 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:18:10.003 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:18:10.060 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:10.092 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:18:10.189 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:18:10.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:18:10.243 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:18:10.267 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:18:10.273 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:10.335 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:18:10.342 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:10.342 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:10.343 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:18:10.343 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:10.343 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:10.343 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:10.375 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:18:10.407 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:10.437 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:18:10.444 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:10.487 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:18:10.587 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:18:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:18:10.677 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:18:10.731 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:18:10.769 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:18:10.825 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:18:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:18:10.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:18:11.058 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-04-26T21:18:11.087 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:11.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:18:11.118 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:18:11.304 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:18:11.324 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:11.369 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:11.369 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:11.369 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:18:11.369 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:11.489 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:18:11.532 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:18:11.570 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:18:11.608 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:18:11.665 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:18:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:18:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:11.717 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:18:11.759 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:18:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:11.771 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:11.771 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:18:11.771 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:11.771 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:11.804 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:18:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:11.855 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:18:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:11.891 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:11.891 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:18:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:11.929 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:18:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:18:12.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:18:12.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:18:12.324 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:18:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:12.469 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:18:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:12.497 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:12.497 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:18:12.497 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:12.498 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:18:12.991 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:13.068 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:13.068 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:13.069 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:18:13.069 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:13.069 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:13.069 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:13.316 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:13.385 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:13.471 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:18:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:18:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:18:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:18:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:18:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:18:13.866 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:18:13.957 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:13.957 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:13.958 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:18:13.958 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:13.958 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:13.958 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:14.101 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:18:14.164 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:18:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:18:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:18:14.239 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:18:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:18:14.472 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:18:14.538 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:18:14.636 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:18:14.740 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:18:14.910 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:18:14.963 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:18:15.004 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:18:15.038 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:18:15.130 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:18:15.278 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:18:15.732 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:15.783 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:15.829 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:18:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:18:15.967 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:17.162 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:18:17.253 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:18:17.260 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:17.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:18:17.312 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:17.358 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:18:17.530 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:18:17.586 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:18:17.694 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:18:17.745 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:18:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:18:17.791 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:18:17.792 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:18:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:18:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:18:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:18:18.134 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:18:18.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:18:18.212 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:18:18.404 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:18.428 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:18:18.572 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:18:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:18:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:18:18.767 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:18:18.797 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:18.866 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:18.867 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:18.867 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:18:18.867 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:18.867 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:18.867 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:18.923 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:18.960 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:19.106 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:18:19.106 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:19.152 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:19.166 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:18:19.203 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:20.172 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:18:20.210 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:20.275 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:20.276 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:20.276 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:18:20.276 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:20.276 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:20.276 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:20.335 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:20.402 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:20.402 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:20.403 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:18:20.403 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:20.478 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:20.514 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:18:20.552 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:20.553 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:20.553 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:18:20.553 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:20.553 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:20.553 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:20.626 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:18:20.723 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:18:20.930 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:18:20.936 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:20.974 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:18:20.974 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:21.009 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:21.009 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:21.009 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:18:21.009 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:21.009 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:21.010 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:21.098 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:21.139 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:18:21.140 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:21.292 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:21.361 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:18:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:18:21.447 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:21.745 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:21.745 INFO:teuthology.orchestra.run.smithi111.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:18:21.745 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:22.063 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:18:22.275 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:18:22.275 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:18:22.275 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:18:22.275 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:22.313 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:18:26.054 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:18:26.112 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:18:26.202 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:18:26.236 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:18:26.279 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:18:26.630 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:18:26.683 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:18:26.736 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:18:27.236 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:18:27.237 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:18:27.238 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:18:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:18:27.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:18:27.252 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:18:27.253 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:18:27.254 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:18:27.255 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:18:27.256 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:18:27.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:18:27.258 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:18:27.259 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:18:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:27.710 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:27.766 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:29.095 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:18:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:18:29.280 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:18:29.446 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:18:29.491 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:18:29.491 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:29.617 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:29.657 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:18:29.657 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:29.736 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:29.736 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.737 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.738 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:18:29.739 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:18:29.740 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:18:29.741 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:18:29.742 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:18:29.743 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:18:29.744 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:18:29.745 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:18:29.746 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:18:29.747 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:18:29.748 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:18:29.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-26T21:18:29.748 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-26T21:18:29.824 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:29.916 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:30.186 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:30.186 INFO:teuthology.orchestra.run.smithi143.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:18:30.186 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:30.343 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-26T21:18:35.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:35.701 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:35.749 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:18:35.749 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:35.749 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:18:35.749 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:36.063 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:18:36.063 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:18:36.064 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:18:36.065 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:18:36.077 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:18:36.078 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:18:36.079 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:18:36.080 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:18:36.081 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:18:36.082 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:18:36.083 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:18:36.084 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:18:36.084 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:18:36.084 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:18:36.084 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:18:36.084 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:18:36.086 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:18:36.087 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:18:37.370 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:18:37.448 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:37.704 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:18:37.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:38.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:18:38.321 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:18:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:18:38.322 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:38.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:18:38.535 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:18:38.611 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-04-26T21:18:38.624 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.625 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.626 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:18:38.627 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:18:38.628 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:18:38.629 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:18:38.630 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:18:38.631 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:18:38.632 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:18:38.634 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:18:38.635 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-04-26T21:18:38.845 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:18:38.900 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:18:38.986 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:18:39.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:18:39.243 DEBUG:teuthology.parallel:result is None 2024-04-26T21:18:39.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:18:39.309 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:18:39.350 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:18:39.391 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:18:39.992 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:40.036 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:18:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:18:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:18:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:42.684 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:18:42.714 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:42.859 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:18:43.035 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:18:43.058 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:43.125 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:18:43.125 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:43.125 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:18:43.126 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:43.126 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:18:43.126 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:43.237 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:18:43.404 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:18:43.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:18:44.504 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:18:44.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:18:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:18:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:44.873 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:18:44.874 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:44.874 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:18:44.874 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:44.874 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:18:44.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:45.246 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:18:45.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:18:50.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:18:50.376 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:18:50.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:18:50.501 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:18:50.549 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:18:50.902 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:18:50.977 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:18:51.004 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:18:51.892 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:51.944 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:52.000 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:18:53.346 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:18:53.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:18:53.534 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:18:53.713 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:18:53.765 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:53.917 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:18:53.964 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:18:53.964 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:54.104 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:54.526 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:18:54.526 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:18:54.527 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:19:00.316 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:19:00.316 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:19:00.316 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:19:00.316 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:19:00.316 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:19:00.317 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:19:00.318 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:19:00.330 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:19:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:19:00.332 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:19:00.333 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:19:00.334 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:19:00.335 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:19:00.336 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:19:00.337 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:19:00.338 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:19:00.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:19:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:19:02.627 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.628 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:19:02.629 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.630 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:19:02.631 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:19:02.632 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:19:02.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:19:02.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:19:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:19:02.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:19:02.638 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:19:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:19:02.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:19:02.639 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-26T21:19:03.227 DEBUG:teuthology.parallel:result is None 2024-04-26T21:19:03.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:03.374 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.428 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.428 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:19:03.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:03.651 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:19:03.723 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.724 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.724 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:19:03.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:03.873 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:19:03.946 INFO:teuthology.orchestra.run.smithi143.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.946 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:19:03.946 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:19:03.948 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:19:03.948 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:19:03.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:19:04.037 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:19:04.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:19:04.130 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:19:04.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:19:04.223 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:19:04.223 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:19:04.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:19:04.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:19:04.399 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:19:04.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:19:04.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:19:04.612 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:19:04.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:19:04.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:19:04.829 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:19:04.829 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:19:04.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:19:04.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:19:05.004 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:19:05.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:19:05.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:19:05.200 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:19:05.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:19:05.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:19:05.416 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:19:05.417 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:19:05.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:19:05.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:19:05.589 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:19:05.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:19:05.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:19:05.733 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:19:05.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:19:05.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:19:05.907 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T21:19:06.004 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T21:19:06.004 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:06.005 INFO:tasks.cephadm:Cluster fsid is 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:19:06.005 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T21:19:06.005 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.b': '172.21.15.111', 'mon.c': '172.21.15.143'} 2024-04-26T21:19:06.005 INFO:tasks.cephadm:First mon is mon.a on smithi097 2024-04-26T21:19:06.005 INFO:tasks.cephadm:First mgr is x 2024-04-26T21:19:06.005 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T21:19:06.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-26T21:19:06.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-04-26T21:19:06.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-04-26T21:19:06.254 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T21:19:06.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:06.541 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T21:19:06.680 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:19:06.681 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T21:19:06.682 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:07.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:19 /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:07.336 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:08.221 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:19 /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:08.222 DEBUG:teuthology.orchestra.run.smithi143:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:09.019 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:19 /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:09.020 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:09.063 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:09.124 DEBUG:teuthology.orchestra.run.smithi143:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:19:09.204 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T21:19:09.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:19:09.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:19:09.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:19:11.164 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:19:11.307 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:19:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:19:47.530 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-04-26T21:19:47.530 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:19:47.530 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:19:47.530 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-04-26T21:19:47.530 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:19:47.531 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-04-26T21:19:47.531 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-04-26T21:19:48.340 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-26T21:19:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:19:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:19:48.340 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-04-26T21:19:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:19:48.341 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-26T21:19:48.341 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-26T21:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:19:51.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-26T21:19:51.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-04-26T21:19:51.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-04-26T21:19:51.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-26T21:19:51.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-04-26T21:19:51.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-04-26T21:19:51.735 INFO:tasks.cephadm:Writing seed config... 2024-04-26T21:19:51.736 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T21:19:51.736 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T21:19:51.737 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T21:19:51.738 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T21:19:51.739 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T21:19:51.740 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-26T21:19:51.740 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:19:51.741 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T21:19:51.786 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9ce681f8-0412-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T21:19:51.786 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T21:19:51.835 DEBUG:teuthology.orchestra.run.smithi097:mgr.x> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T21:19:51.879 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T21:19:51.879 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:19:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:19:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', '9ce681f8-0412-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2024-04-26T21:19:52.140 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T21:19:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2024-04-26T21:19:52.193 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:19:52.193 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2024-04-26T21:19:52.193 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2024-04-26T21:19:52.208 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:19:52.208 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:19:52.220 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:19:52.221 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:19:52.235 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-04-26T21:19:52.248 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-04-26T21:19:52.248 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:19:52.248 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2024-04-26T21:19:52.298 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:19:52.298 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T21:19:52.298 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2024-04-26T21:19:52.298 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2024-04-26T21:19:52.315 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:19:52.316 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:19:52.335 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:19:52.335 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:19:52.361 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-04-26T21:19:52.383 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-04-26T21:19:52.383 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:19:52.383 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2024-04-26T21:19:52.384 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:19:52.384 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 139909575670944 on /run/cephadm/9ce681f8-0412-11ef-bc93-c7b262605968.lock 2024-04-26T21:19:52.385 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 139909575670944 acquired on /run/cephadm/9ce681f8-0412-11ef-bc93-c7b262605968.lock 2024-04-26T21:19:52.386 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2024-04-26T21:19:52.387 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2024-04-26T21:19:52.387 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-04-26T21:19:52.402 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.97 metric 100 2024-04-26T21:19:52.402 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2024-04-26T21:19:52.415 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T21:19:52.415 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-26T21:19:52.415 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-26T21:19:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-04-26T21:19:52.429 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T21:19:52.432 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T21:19:52.434 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:19:53.056 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T21:19:54.283 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:19:54.283 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:19:54.283 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T21:20:05.627 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2024-04-26T21:20:05.627 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2024-04-26T21:20:06.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCGGixmVdxOBRAA2CYsLhevDSb0exaUovAdyQ== 2024-04-26T21:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCHGixmcKnrDRAAF8iEF8QvCE14UntRlEY2cg== 2024-04-26T21:20:08.982 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCIGixmv9y9ERAAfbWc3wMN451ASEmhT8nOXg== 2024-04-26T21:20:08.987 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2024-04-26T21:20:10.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:20:10.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T21:20:10.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:20:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2024-04-26T21:20:11.854 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2024-04-26T21:20:12.222 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T21:20:12.464 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T21:20:12.771 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target. 2024-04-26T21:20:12.772 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target. 2024-04-26T21:20:13.167 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:20:13.167 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T21:20:13.448 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target.wants/ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968@.service. 2024-04-26T21:20:14.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:13 smithi097 podman[27739]: 2024-04-26 21:20:13.743636607 +0000 UTC m=+0.031698412 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:20:14.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 podman[27739]: 2024-04-26 21:20:14.094906582 +0000 UTC m=+0.382968428 container init 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:20:14.214 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:20:14.215 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-26T21:20:14.228 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:20:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:20:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-26T21:20:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:20:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2024-04-26T21:20:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2024-04-26T21:20:14.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 podman[27739]: 2024-04-26 21:20:14.105423415 +0000 UTC m=+0.393485281 container start 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:20:14.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 bash[27739]: 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e 2024-04-26T21:20:14.363 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 ceph-mon[27757]: mkfs 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:14.363 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 ceph-mon[27757]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:20:14.363 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:14 smithi097 systemd[1]: Started Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: monmap epoch 1 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: last_changed 2024-04-26T21:20:09.486150+0000 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: min_mon_release 19 (squid) 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: election_strategy: 1 2024-04-26T21:20:15.520 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:20:15.521 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: fsmap 2024-04-26T21:20:15.521 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:20:15.521 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: mgrmap e1: no daemons active 2024-04-26T21:20:15.521 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:15 smithi097 ceph-mon[27757]: from='client.? 172.21.15.97:0/1892437493' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T21:20:15.528 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.719883s) 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:15.535 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.809+0000 7fa3edc00700 1 Processor -- start 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.810+0000 7fa3edc00700 1 -- start start 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.810+0000 7fa3edc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.810+0000 7fa3edc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810ba20 con 0x7fa3e810b030 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.811+0000 7fa3e6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.811+0000 7fa3e6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58064/0 (socket says 172.21.15.97:58064) 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.811+0000 7fa3e6a00700 1 -- 172.21.15.97:0/3035634335 learned_addr learned my addr 172.21.15.97:0/3035634335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.812+0000 7fa3e6a00700 1 -- 172.21.15.97:0/3035634335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e810c230 con 0x7fa3e810b030 2024-04-26T21:20:15.536 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.812+0000 7fa3e6a00700 1 --2- 172.21.15.97:0/3035634335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80098a0 tx=0x7fa3d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd4cdded8dd643c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.812+0000 7fa3e6000700 1 -- 172.21.15.97:0/3035634335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d8004160 con 0x7fa3e810b030 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.812+0000 7fa3e6000700 1 -- 172.21.15.97:0/3035634335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d8004300 con 0x7fa3e810b030 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3e6000700 1 -- 172.21.15.97:0/3035634335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d80036a0 con 0x7fa3e810b030 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 -- 172.21.15.97:0/3035634335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 msgr2=0x7fa3e810b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 --2- 172.21.15.97:0/3035634335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80098a0 tx=0x7fa3d8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 -- 172.21.15.97:0/3035634335 shutdown_connections 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 --2- 172.21.15.97:0/3035634335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e810b450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 -- 172.21.15.97:0/3035634335 >> 172.21.15.97:0/3035634335 conn(0x7fa3e807e440 msgr2=0x7fa3e807e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 -- 172.21.15.97:0/3035634335 shutdown_connections 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.813+0000 7fa3edc00700 1 -- 172.21.15.97:0/3035634335 wait complete. 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3edc00700 1 Processor -- start 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3edc00700 1 -- start start 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3edc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3edc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810c760 con 0x7fa3e810b030 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3e6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:15.537 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3e6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58072/0 (socket says 172.21.15.97:58072) 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.815+0000 7fa3e6a00700 1 -- 172.21.15.97:0/1892437493 learned_addr learned my addr 172.21.15.97:0/1892437493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3e6a00700 1 -- 172.21.15.97:0/1892437493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e819ec90 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3e6a00700 1 --2- 172.21.15.97:0/1892437493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8000c00 tx=0x7fa3d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d8003db0 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d8011630 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d8011950 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.816+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e819ef20 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.817+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a1c10 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.817+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3d801a460 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.817+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3d801b780 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.818+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8064a80 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.821+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa3d801b980 con 0x7fa3e810b030 2024-04-26T21:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.900+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa3e802e6e0 con 0x7fa3e810b030 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.901+0000 7fa3dfe00700 1 -- 172.21.15.97:0/1892437493 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa3d801bb60 con 0x7fa3e810b030 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 msgr2=0x7fa3e819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 --2- 172.21.15.97:0/1892437493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8000c00 tx=0x7fa3d8004750 comp rx=0 tx=0).stop 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 shutdown_connections 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 --2- 172.21.15.97:0/1892437493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e810b030 0x7fa3e819e750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 >> 172.21.15.97:0/1892437493 conn(0x7fa3e807e440 msgr2=0x7fa3e81953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 shutdown_connections 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:14.902+0000 7fa3edc00700 1 -- 172.21.15.97:0/1892437493 wait complete. 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2024-04-26T21:20:15.539 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T21:20:16.526 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:16 smithi097 ceph-mon[27757]: from='client.? 172.21.15.97:0/534001923' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:20:16.527 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:16 smithi097 ceph-mon[27757]: from='client.? 172.21.15.97:0/534001923' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T21:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.113+0000 7f7c48c00700 1 Processor -- start 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.113+0000 7f7c48c00700 1 -- start start 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.113+0000 7f7c48c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.113+0000 7f7c48c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4407ce30 con 0x7f7c441098d0 2024-04-26T21:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.114+0000 7f7c41e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.114+0000 7f7c41e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58074/0 (socket says 172.21.15.97:58074) 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.114+0000 7f7c41e00700 1 -- 172.21.15.97:0/1950688807 learned_addr learned my addr 172.21.15.97:0/1950688807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.114+0000 7f7c41e00700 1 -- 172.21.15.97:0/1950688807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4410c200 con 0x7f7c441098d0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.115+0000 7f7c41e00700 1 --2- 172.21.15.97:0/1950688807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7c340098a0 tx=0x7f7c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7599d93db8a546cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.115+0000 7f7c40a00700 1 -- 172.21.15.97:0/1950688807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c34004160 con 0x7f7c441098d0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.115+0000 7f7c40a00700 1 -- 172.21.15.97:0/1950688807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c34004300 con 0x7f7c441098d0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.115+0000 7f7c40a00700 1 -- 172.21.15.97:0/1950688807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c340036a0 con 0x7f7c441098d0 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 -- 172.21.15.97:0/1950688807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 msgr2=0x7f7c4410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 --2- 172.21.15.97:0/1950688807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7c340098a0 tx=0x7f7c34009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 -- 172.21.15.97:0/1950688807 shutdown_connections 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 --2- 172.21.15.97:0/1950688807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4410bcc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 -- 172.21.15.97:0/1950688807 >> 172.21.15.97:0/1950688807 conn(0x7f7c44103320 msgr2=0x7f7c44105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 -- 172.21.15.97:0/1950688807 shutdown_connections 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.116+0000 7f7c48c00700 1 -- 172.21.15.97:0/1950688807 wait complete. 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.117+0000 7f7c48c00700 1 Processor -- start 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.117+0000 7f7c48c00700 1 -- start start 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.117+0000 7f7c48c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.117+0000 7f7c48c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410c6d0 con 0x7f7c441098d0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c41e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c41e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58088/0 (socket says 172.21.15.97:58088) 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c41e00700 1 -- 172.21.15.97:0/534001923 learned_addr learned my addr 172.21.15.97:0/534001923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c41e00700 1 -- 172.21.15.97:0/534001923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419caa0 con 0x7f7c441098d0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c41e00700 1 --2- 172.21.15.97:0/534001923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.118+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c34003db0 con 0x7f7c441098d0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c34011630 con 0x7f7c441098d0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4419cd30 con 0x7f7c441098d0 2024-04-26T21:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c34011950 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4419fa20 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c3401a460 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.119+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c3401b7d0 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.120+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44064a80 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.123+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7c3401ba70 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.198+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7c4402e790 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.206+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+517 (secure 0 0 0) 0x7f7c3401bc50 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c3aa00700 1 -- 172.21.15.97:0/534001923 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7c3401ad50 con 0x7f7c441098d0 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 msgr2=0x7f7c4419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 --2- 172.21.15.97:0/534001923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c34004750 comp rx=0 tx=0).stop 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 shutdown_connections 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 --2- 172.21.15.97:0/534001923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c441098d0 0x7f7c4419c560 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 >> 172.21.15.97:0/534001923 conn(0x7f7c44103320 msgr2=0x7f7c441055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 shutdown_connections 2024-04-26T21:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:16.208+0000 7f7c48c00700 1 -- 172.21.15.97:0/534001923 wait complete. 2024-04-26T21:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2024-04-26T21:20:17.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:17 smithi097 ceph-mon[27757]: from='client.? 172.21.15.97:0/2322356798' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:18.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.438+0000 7fb716200700 1 Processor -- start 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.439+0000 7fb716200700 1 -- start start 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.439+0000 7fb716200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.439+0000 7fb716200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb710112610 con 0x7fb71010fc50 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.439+0000 7fb70f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.440+0000 7fb70f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58090/0 (socket says 172.21.15.97:58090) 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.440+0000 7fb70f400700 1 -- 172.21.15.97:0/3078538269 learned_addr learned my addr 172.21.15.97:0/3078538269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.440+0000 7fb70f400700 1 -- 172.21.15.97:0/3078538269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb710112790 con 0x7fb71010fc50 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.440+0000 7fb70f400700 1 --2- 172.21.15.97:0/3078538269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7000098a0 tx=0x7fb700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f8cd85f918b6dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb70e000700 1 -- 172.21.15.97:0/3078538269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700004160 con 0x7fb71010fc50 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb70e000700 1 -- 172.21.15.97:0/3078538269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb700004300 con 0x7fb71010fc50 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb70e000700 1 -- 172.21.15.97:0/3078538269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7000119e0 con 0x7fb71010fc50 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb716200700 1 -- 172.21.15.97:0/3078538269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 msgr2=0x7fb710112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb716200700 1 --2- 172.21.15.97:0/3078538269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7000098a0 tx=0x7fb700009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb716200700 1 -- 172.21.15.97:0/3078538269 shutdown_connections 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb716200700 1 --2- 172.21.15.97:0/3078538269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb710112040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.441+0000 7fb716200700 1 -- 172.21.15.97:0/3078538269 >> 172.21.15.97:0/3078538269 conn(0x7fb71007e460 msgr2=0x7fb710080880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.442+0000 7fb716200700 1 -- 172.21.15.97:0/3078538269 shutdown_connections 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.442+0000 7fb716200700 1 -- 172.21.15.97:0/3078538269 wait complete. 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb716200700 1 Processor -- start 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb716200700 1 -- start start 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb716200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb716200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb710113760 con 0x7fb71010fc50 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb70f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb70f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58098/0 (socket says 172.21.15.97:58098) 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.443+0000 7fb70f400700 1 -- 172.21.15.97:0/2322356798 learned_addr learned my addr 172.21.15.97:0/2322356798 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb70f400700 1 -- 172.21.15.97:0/2322356798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7101a3820 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb70f400700 1 --2- 172.21.15.97:0/2322356798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb7000116a0 tx=0x7fb7000046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700003b70 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb70001a8a0 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7101a3ab0 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700024890 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.444+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101a67a0 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.445+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb700022020 con 0x7fb71010fc50 2024-04-26T21:20:18.220 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.445+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7000193c0 con 0x7fb71010fc50 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.446+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb710064a80 con 0x7fb71010fc50 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.448+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb70001aa50 con 0x7fb71010fc50 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.525+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb7101a6970 con 0x7fb71010fc50 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.526+0000 7fb707e00700 1 -- 172.21.15.97:0/2322356798 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb700004300 con 0x7fb71010fc50 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 msgr2=0x7fb7101a32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 --2- 172.21.15.97:0/2322356798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb7000116a0 tx=0x7fb7000046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 shutdown_connections 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 --2- 172.21.15.97:0/2322356798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb71010fc50 0x7fb7101a32e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 >> 172.21.15.97:0/2322356798 conn(0x7fb71007e460 msgr2=0x7fb71007ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 shutdown_connections 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:17.527+0000 7fb716200700 1 -- 172.21.15.97:0/2322356798 wait complete. 2024-04-26T21:20:18.221 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2024-04-26T21:20:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:18 smithi097 systemd[1]: Stopping Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:20:18.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a[27753]: 2024-04-26T21:20:18.729+0000 7f4838e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:20:18.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a[27753]: 2024-04-26T21:20:18.729+0000 7f4838e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T21:20:18.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:18 smithi097 podman[28001]: 2024-04-26 21:20:18.750506047 +0000 UTC m=+0.161300139 container died 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T21:20:19.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 podman[28001]: 2024-04-26 21:20:19.112682199 +0000 UTC m=+0.523476350 container cleanup 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T21:20:19.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 bash[28001]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:20:19.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 podman[28015]: 2024-04-26 21:20:19.305166687 +0000 UTC m=+0.538403718 container remove 4a400d649c97b151fd1fa89e32ffaa17cc5679687319b2a0a5be5090546c1a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T21:20:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 systemd[1]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T21:20:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 systemd[1]: Stopped Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:20:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 systemd[1]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.862s CPU time. 2024-04-26T21:20:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:19 smithi097 systemd[1]: Starting Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:20:20.403 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 podman[28107]: 2024-04-26 21:20:20.055551787 +0000 UTC m=+0.040907590 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:20:20.404 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 podman[28107]: 2024-04-26 21:20:20.156173646 +0000 UTC m=+0.141529487 container create 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:20:20.501 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T21:20:20.658 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 podman[28107]: 2024-04-26 21:20:20.402756161 +0000 UTC m=+0.388111997 container init 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 podman[28107]: 2024-04-26 21:20:20.412442184 +0000 UTC m=+0.397798032 container start 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: pidfile_write: ignore empty --pid-file 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: load: jerasure load: lrc 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Git sha 0 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: DB SUMMARY 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: DB Session ID: 1H7NZ968OARIYKFL2MM1 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:20:20.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104906 ; 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.env: 0x556793261cc0 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.info_log: 0x5567a256ddc0 2024-04-26T21:20:20.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.statistics: (nil) 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.use_fsync: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.db_log_dir: 2024-04-26T21:20:20.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.wal_dir: 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.write_buffer_manager: 0x5567a253f0e0 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:20:20.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.unordered_write: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.row_cache: None 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.wal_filter: None 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:20:20.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.wal_compression: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:20:20.664 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_open_files: -1 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:20:20.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Compression algorithms supported: 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kZSTD supported: 0 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:20:20.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.merge_operator: 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_filter: None 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5567a256df80) 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2024-04-26T21:20:20.667 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x5567a25711f0 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2024-04-26T21:20:20.668 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:20:20.669 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression: NoCompression 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.num_levels: 7 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:20:20.670 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:20:20.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:20:20.672 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:20:20.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:20:20.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:20:20.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:20:20.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:20:20.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:20:20.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:20:20.675 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.ttl: 2592000 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:20:20.676 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 455ce1bc-387e-4eb5-b337-35f5aa1fd254 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166420465912, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166420467778, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97952, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14143, "raw_average_key_size": 49, "raw_value_size": 90268, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714166420, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "455ce1bc-387e-4eb5-b337-35f5aa1fd254", "db_session_id": "1H7NZ968OARIYKFL2MM1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166420467975, "job": 1, "event": "recovery_finished"} 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5567a268e000 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: DB pointer 0x5567a267a000 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2024-04-26T21:20:20.677 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 99.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 99.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:20:20.678 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 14.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 14.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:20:20.679 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x5567a25711f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???) e1 preinit fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).mds e1 new map 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).mds e1 print_map 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2024-04-26T21:20:20.680 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 bash[28107]: 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 systemd[1]: Started Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: monmap epoch 1 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: last_changed 2024-04-26T21:20:09.486150+0000 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: min_mon_release 19 (squid) 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: election_strategy: 1 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:20:20.681 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: fsmap 2024-04-26T21:20:20.682 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:20:20.682 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:20 smithi097 ceph-mon[28127]: mgrmap e1: no daemons active 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.161+0000 7ff6a6800700 1 Processor -- start 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.161+0000 7ff6a6800700 1 -- start start 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.162+0000 7ff6a6800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.162+0000 7ff6a6800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a007cdf0 con 0x7ff6a0109960 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.163+0000 7ff69f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.163+0000 7ff69f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33198/0 (socket says 172.21.15.97:33198) 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.163+0000 7ff69f400700 1 -- 172.21.15.97:0/4001111468 learned_addr learned my addr 172.21.15.97:0/4001111468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.163+0000 7ff69f400700 1 -- 172.21.15.97:0/4001111468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a010c290 con 0x7ff6a0109960 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.163+0000 7ff69f400700 1 --2- 172.21.15.97:0/4001111468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098a0 tx=0x7ff690009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c9f2cba1a0050e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.811 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.164+0000 7ff69e000700 1 -- 172.21.15.97:0/4001111468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690004160 con 0x7ff6a0109960 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.164+0000 7ff69e000700 1 -- 172.21.15.97:0/4001111468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff690004300 con 0x7ff6a0109960 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.164+0000 7ff69e000700 1 -- 172.21.15.97:0/4001111468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6900119e0 con 0x7ff6a0109960 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.164+0000 7ff6a6800700 1 -- 172.21.15.97:0/4001111468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 msgr2=0x7ff6a010bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.164+0000 7ff6a6800700 1 --2- 172.21.15.97:0/4001111468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098a0 tx=0x7ff690009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.165+0000 7ff6a6800700 1 -- 172.21.15.97:0/4001111468 shutdown_connections 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.165+0000 7ff6a6800700 1 --2- 172.21.15.97:0/4001111468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a010bd50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.165+0000 7ff6a6800700 1 -- 172.21.15.97:0/4001111468 >> 172.21.15.97:0/4001111468 conn(0x7ff6a0103370 msgr2=0x7ff6a0105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.165+0000 7ff6a6800700 1 -- 172.21.15.97:0/4001111468 shutdown_connections 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.165+0000 7ff6a6800700 1 -- 172.21.15.97:0/4001111468 wait complete. 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff6a6800700 1 Processor -- start 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff6a6800700 1 -- start start 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff6a6800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff6a6800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a010d1f0 con 0x7ff6a0109960 2024-04-26T21:20:21.812 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff69f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff69f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33210/0 (socket says 172.21.15.97:33210) 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.166+0000 7ff69f400700 1 -- 172.21.15.97:0/1394933253 learned_addr learned my addr 172.21.15.97:0/1394933253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff69f400700 1 -- 172.21.15.97:0/1394933253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a019f400 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff69f400700 1 --2- 172.21.15.97:0/1394933253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff690009130 tx=0x7ff690011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690003b20 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a019f690 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff69001a8a0 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690024810 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.167+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a01a2380 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.168+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff690022020 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.168+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff69002d990 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.169+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0064a80 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.172+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff69001ad70 con 0x7ff6a0109960 2024-04-26T21:20:21.813 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.244+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff6a01a2550 con 0x7ff6a0109960 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.248+0000 7ff697e00700 1 -- 172.21.15.97:0/1394933253 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff69001aa50 con 0x7ff6a0109960 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 msgr2=0x7ff6a019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 --2- 172.21.15.97:0/1394933253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff690009130 tx=0x7ff690011a60 comp rx=0 tx=0).stop 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 shutdown_connections 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 --2- 172.21.15.97:0/1394933253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a0109960 0x7ff6a019eec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 >> 172.21.15.97:0/1394933253 conn(0x7ff6a0103370 msgr2=0x7ff6a0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 shutdown_connections 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:21.250+0000 7ff6a6800700 1 -- 172.21.15.97:0/1394933253 wait complete. 2024-04-26T21:20:21.814 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:20:21.832 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2024-04-26T21:20:21.834 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T21:20:21.835 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T21:20:22.261 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:20:22.261 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T21:20:22.531 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968.target.wants/ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9ce681f8-0412-11ef-bc93-c7b262605968@.service. 2024-04-26T21:20:22.560 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1394933253' entity='client.admin' 2024-04-26T21:20:22.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:22 smithi097 systemd[1]: Starting Ceph mgr.x for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:20:23.165 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:22 smithi097 podman[28353]: 2024-04-26 21:20:22.807546294 +0000 UTC m=+0.031903231 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:20:23.165 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:22 smithi097 podman[28353]: 2024-04-26 21:20:22.934711591 +0000 UTC m=+0.159068565 container create 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T21:20:23.165 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 podman[28353]: 2024-04-26 21:20:23.163283969 +0000 UTC m=+0.387640949 container init 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T21:20:23.278 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:20:23.278 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-26T21:20:23.297 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:20:23.297 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:20:23.298 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-26T21:20:23.298 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:20:23.320 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:20:23.321 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-26T21:20:23.339 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:20:23.339 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:20:23.339 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-26T21:20:23.340 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T21:20:23.340 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2024-04-26T21:20:23.340 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2024-04-26T21:20:23.490 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 podman[28353]: 2024-04-26 21:20:23.173527356 +0000 UTC m=+0.397884337 container start 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T21:20:23.490 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 bash[28353]: 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 2024-04-26T21:20:23.490 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 systemd[1]: Started Ceph mgr.x for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:20:23.490 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:23.411+0000 7fee56ae9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:20:23.783 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:23.663+0000 7fee56ae9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:20:24.044 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:23 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:23.796+0000 7fee56ae9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:20:24.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:24 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1901821895' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:20:24.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:24 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:24.215+0000 7fee56ae9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:20:24.678 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:24 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:24.329+0000 7fee56ae9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:20:24.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:20:24.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:20:24.692 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:20:24.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:20:24.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:20:14.184199+0000", 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.934+0000 7f68bfe00700 1 Processor -- start 2024-04-26T21:20:24.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.934+0000 7f68bfe00700 1 -- start start 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.935+0000 7f68bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.935+0000 7f68bfe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68c007cdd0 con 0x7f68c0109720 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.935+0000 7f68bea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.935+0000 7f68bea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33232/0 (socket says 172.21.15.97:33232) 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.935+0000 7f68bea00700 1 -- 172.21.15.97:0/2076873090 learned_addr learned my addr 172.21.15.97:0/2076873090 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.936+0000 7f68bea00700 1 -- 172.21.15.97:0/2076873090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c010c050 con 0x7f68c0109720 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.936+0000 7f68bea00700 1 --2- 172.21.15.97:0/2076873090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098a0 tx=0x7f68b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb0fd2f72b9d278 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.937+0000 7f68bd600700 1 -- 172.21.15.97:0/2076873090 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b0004160 con 0x7f68c0109720 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.937+0000 7f68bd600700 1 -- 172.21.15.97:0/2076873090 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f68b0004300 con 0x7f68c0109720 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.937+0000 7f68bd600700 1 -- 172.21.15.97:0/2076873090 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b00119e0 con 0x7f68c0109720 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.937+0000 7f68bfe00700 1 -- 172.21.15.97:0/2076873090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 msgr2=0x7f68c010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.937+0000 7f68bfe00700 1 --2- 172.21.15.97:0/2076873090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098a0 tx=0x7f68b0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.938+0000 7f68bfe00700 1 -- 172.21.15.97:0/2076873090 shutdown_connections 2024-04-26T21:20:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.938+0000 7f68bfe00700 1 --2- 172.21.15.97:0/2076873090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c010bb10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.938+0000 7f68bfe00700 1 -- 172.21.15.97:0/2076873090 >> 172.21.15.97:0/2076873090 conn(0x7f68c0103110 msgr2=0x7f68c0105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.938+0000 7f68bfe00700 1 -- 172.21.15.97:0/2076873090 shutdown_connections 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.938+0000 7f68bfe00700 1 -- 172.21.15.97:0/2076873090 wait complete. 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.939+0000 7f68bfe00700 1 Processor -- start 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.939+0000 7f68bfe00700 1 -- start start 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.940+0000 7f68bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.940+0000 7f68bfe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68c010cf60 con 0x7f68c0109720 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.940+0000 7f68bea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.940+0000 7f68bea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33234/0 (socket says 172.21.15.97:33234) 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.940+0000 7f68bea00700 1 -- 172.21.15.97:0/1901821895 learned_addr learned my addr 172.21.15.97:0/1901821895 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68bea00700 1 -- 172.21.15.97:0/1901821895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c019f2d0 con 0x7f68c0109720 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68bea00700 1 --2- 172.21.15.97:0/1901821895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68b00116a0 tx=0x7f68b00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b0003b70 con 0x7f68c0109720 2024-04-26T21:20:24.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f68b001a8a0 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b0024890 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.941+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c019f560 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.942+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c01a2210 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.942+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f68b0022020 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.942+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f68b00193c0 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.943+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c0064a80 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:23.946+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f68b001aa50 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.025+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f68c01a2480 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.025+0000 7f68b7400700 1 -- 172.21.15.97:0/1901821895 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f68b0004300 con 0x7f68c0109720 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 msgr2=0x7f68c019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 --2- 172.21.15.97:0/1901821895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68b00116a0 tx=0x7f68b00046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 shutdown_connections 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 --2- 172.21.15.97:0/1901821895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68c0109720 0x7f68c019ed90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 >> 172.21.15.97:0/1901821895 conn(0x7f68c0103110 msgr2=0x7f68c0103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:24.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 shutdown_connections 2024-04-26T21:20:24.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:24.027+0000 7f68bfe00700 1 -- 172.21.15.97:0/1901821895 wait complete. 2024-04-26T21:20:24.706 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2024-04-26T21:20:26.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:25 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:25.904+0000 7fee56ae9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:20:26.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:26.126+0000 7fee56ae9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:20:26.725 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:26.352+0000 7fee56ae9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:20:26.725 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:26.724+0000 7fee56ae9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:20:26.996 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:26.849+0000 7fee56ae9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:20:27.293 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:27 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:27.108+0000 7fee56ae9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:20:27.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:27 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4111734764' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:20:27.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:27 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:27.362+0000 7fee56ae9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:20:28.077 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:28.078 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:20:28.079 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:20:28.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:20:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:20:14.184199+0000", 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.291+0000 7f9221000700 1 Processor -- start 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.292+0000 7f9221000700 1 -- start start 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.292+0000 7f9221000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.292+0000 7f9221000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f921c109830 con 0x7f921c108ed0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.293+0000 7f9219e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.293+0000 7f9219e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33248/0 (socket says 172.21.15.97:33248) 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.293+0000 7f9219e00700 1 -- 172.21.15.97:0/3292156380 learned_addr learned my addr 172.21.15.97:0/3292156380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.293+0000 7f9219e00700 1 -- 172.21.15.97:0/3292156380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c109f90 con 0x7f921c108ed0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.293+0000 7f9219e00700 1 --2- 172.21.15.97:0/3292156380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098a0 tx=0x7f920c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfdae4a47869a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.294+0000 7f9218a00700 1 -- 172.21.15.97:0/3292156380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c004160 con 0x7f921c108ed0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.294+0000 7f9218a00700 1 -- 172.21.15.97:0/3292156380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f920c004300 con 0x7f921c108ed0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.294+0000 7f9218a00700 1 -- 172.21.15.97:0/3292156380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c0119e0 con 0x7f921c108ed0 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 -- 172.21.15.97:0/3292156380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 msgr2=0x7f921c1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:28.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 --2- 172.21.15.97:0/3292156380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098a0 tx=0x7f920c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 -- 172.21.15.97:0/3292156380 shutdown_connections 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 --2- 172.21.15.97:0/3292156380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c1092f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 -- 172.21.15.97:0/3292156380 >> 172.21.15.97:0/3292156380 conn(0x7f921c104490 msgr2=0x7f921c1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 -- 172.21.15.97:0/3292156380 shutdown_connections 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.295+0000 7f9221000700 1 -- 172.21.15.97:0/3292156380 wait complete. 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.296+0000 7f9221000700 1 Processor -- start 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.296+0000 7f9221000700 1 -- start start 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.296+0000 7f9221000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.296+0000 7f9221000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f921c10d250 con 0x7f921c108ed0 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.297+0000 7f9219e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.297+0000 7f9219e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33252/0 (socket says 172.21.15.97:33252) 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.297+0000 7f9219e00700 1 -- 172.21.15.97:0/4111734764 learned_addr learned my addr 172.21.15.97:0/4111734764 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.297+0000 7f9219e00700 1 -- 172.21.15.97:0/4111734764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c07f030 con 0x7f921c108ed0 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.297+0000 7f9219e00700 1 --2- 172.21.15.97:0/4111734764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f920c0116a0 tx=0x7f920c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c003b70 con 0x7f921c108ed0 2024-04-26T21:20:28.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c07d220 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f920c01a8a0 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c024890 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c07d760 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f920c022020 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.298+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f920c0193c0 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.299+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921c064a80 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.302+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f920c01aa50 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.380+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f921c07dc90 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.381+0000 7f9212a00700 1 -- 172.21.15.97:0/4111734764 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f920c032030 con 0x7f921c108ed0 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 msgr2=0x7f921c07eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 --2- 172.21.15.97:0/4111734764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f920c0116a0 tx=0x7f920c0046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 shutdown_connections 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 --2- 172.21.15.97:0/4111734764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f921c108ed0 0x7f921c07eaf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:28.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 >> 172.21.15.97:0/4111734764 conn(0x7f921c104490 msgr2=0x7f921c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:28.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 shutdown_connections 2024-04-26T21:20:28.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:27.383+0000 7f9221000700 1 -- 172.21.15.97:0/4111734764 wait complete. 2024-04-26T21:20:28.094 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2024-04-26T21:20:28.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:28 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:28.180+0000 7fee56ae9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:20:29.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:28 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:28.725+0000 7fee56ae9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:20:29.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:28 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:28.850+0000 7fee56ae9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:20:29.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:29 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:29.109+0000 7fee56ae9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:20:30.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:29 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:29.774+0000 7fee56ae9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:20:30.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:29 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:29.898+0000 7fee56ae9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:20:30.529 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:30 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:30.219+0000 7fee56ae9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:20:30.529 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:30 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:30.400+0000 7fee56ae9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:20:30.529 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:30 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:30.527+0000 7fee56ae9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:20:30.788 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:30 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:30.683+0000 7fee56ae9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:20:31.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:30 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3296195616' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:20:31.331 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:31 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:31.072+0000 7fee56ae9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:20:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:20:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:20:31.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:20:31.458 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:20:14.184199+0000", 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:31.459 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.696+0000 7fc07ea00700 1 Processor -- start 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.696+0000 7fc07ea00700 1 -- start start 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc07ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc07ea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc078109830 con 0x7fc078108ed0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc077e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc077e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33262/0 (socket says 172.21.15.97:33262) 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc077e00700 1 -- 172.21.15.97:0/2814204296 learned_addr learned my addr 172.21.15.97:0/2814204296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.697+0000 7fc077e00700 1 -- 172.21.15.97:0/2814204296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc078109f90 con 0x7fc078108ed0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.698+0000 7fc077e00700 1 --2- 172.21.15.97:0/2814204296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0680098a0 tx=0x7fc068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94e4ce581ff6c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.698+0000 7fc076a00700 1 -- 172.21.15.97:0/2814204296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068004160 con 0x7fc078108ed0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.698+0000 7fc076a00700 1 -- 172.21.15.97:0/2814204296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc068004300 con 0x7fc078108ed0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.698+0000 7fc076a00700 1 -- 172.21.15.97:0/2814204296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0680119e0 con 0x7fc078108ed0 2024-04-26T21:20:31.460 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 -- 172.21.15.97:0/2814204296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 msgr2=0x7fc0781092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 --2- 172.21.15.97:0/2814204296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0680098a0 tx=0x7fc068009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 -- 172.21.15.97:0/2814204296 shutdown_connections 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 --2- 172.21.15.97:0/2814204296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc0781092f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 -- 172.21.15.97:0/2814204296 >> 172.21.15.97:0/2814204296 conn(0x7fc078104490 msgr2=0x7fc0781068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 -- 172.21.15.97:0/2814204296 shutdown_connections 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.699+0000 7fc07ea00700 1 -- 172.21.15.97:0/2814204296 wait complete. 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.700+0000 7fc07ea00700 1 Processor -- start 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.700+0000 7fc07ea00700 1 -- start start 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc07ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc07ea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc07810d250 con 0x7fc078108ed0 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc077e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc077e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33272/0 (socket says 172.21.15.97:33272) 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc077e00700 1 -- 172.21.15.97:0/3296195616 learned_addr learned my addr 172.21.15.97:0/3296195616 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.701+0000 7fc077e00700 1 -- 172.21.15.97:0/3296195616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07819f3f0 con 0x7fc078108ed0 2024-04-26T21:20:31.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc077e00700 1 --2- 172.21.15.97:0/3296195616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0680116a0 tx=0x7fc0680046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068003b70 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc06801a8a0 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07819f680 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068024890 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a2370 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc068022020 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.702+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc0680193c0 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.703+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078064a80 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.708+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc06801aa50 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.787+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc0781a28c0 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.788+0000 7fc074c00700 1 -- 172.21.15.97:0/3296195616 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc06801a430 con 0x7fc078108ed0 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 msgr2=0x7fc07819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 --2- 172.21.15.97:0/3296195616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0680116a0 tx=0x7fc0680046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 shutdown_connections 2024-04-26T21:20:31.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 --2- 172.21.15.97:0/3296195616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc078108ed0 0x7fc07819eeb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:31.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 >> 172.21.15.97:0/3296195616 conn(0x7fc078104490 msgr2=0x7fc078104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:31.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 shutdown_connections 2024-04-26T21:20:31.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:30.789+0000 7fc07ea00700 1 -- 172.21.15.97:0/3296195616 wait complete. 2024-04-26T21:20:31.463 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2024-04-26T21:20:32.121 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:31 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:31.829+0000 7fee56ae9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:20:32.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:32.118+0000 7fee56ae9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:20:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: Activating manager daemon x 2024-04-26T21:20:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: mgrmap e2: x(active, starting, since 0.0114318s) 2024-04-26T21:20:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:20:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: Manager daemon x is now available 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' 2024-04-26T21:20:32.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:32 smithi097 ceph-mon[28127]: from='mgr.14100 172.21.15.97:0/152054114' entity='mgr.x' 2024-04-26T21:20:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:34 smithi097 ceph-mon[28127]: mgrmap e3: x(active, since 1.02239s) 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:20:34.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:35.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:20:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:35.005 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:20:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:20:35.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:20:35.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:35.014 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.014 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:20:35.014 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:20:35.014 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:20:14.184199+0000", 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.078+0000 7efd7ca00700 1 Processor -- start 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.078+0000 7efd7ca00700 1 -- start start 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.079+0000 7efd7ca00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.079+0000 7efd7ca00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd7807ce30 con 0x7efd781098d0 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.080+0000 7efd75e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.080+0000 7efd75e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47690/0 (socket says 172.21.15.97:47690) 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.080+0000 7efd75e00700 1 -- 172.21.15.97:0/1498909184 learned_addr learned my addr 172.21.15.97:0/1498909184 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.080+0000 7efd75e00700 1 -- 172.21.15.97:0/1498909184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7810c200 con 0x7efd781098d0 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.081+0000 7efd75e00700 1 --2- 172.21.15.97:0/1498909184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd680098a0 tx=0x7efd68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5256c7753527e1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:35.015 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.081+0000 7efd74a00700 1 -- 172.21.15.97:0/1498909184 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd68004160 con 0x7efd781098d0 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.081+0000 7efd74a00700 1 -- 172.21.15.97:0/1498909184 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efd68004300 con 0x7efd781098d0 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.081+0000 7efd74a00700 1 -- 172.21.15.97:0/1498909184 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd680119e0 con 0x7efd781098d0 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 -- 172.21.15.97:0/1498909184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 msgr2=0x7efd7810bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 --2- 172.21.15.97:0/1498909184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd680098a0 tx=0x7efd68009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 -- 172.21.15.97:0/1498909184 shutdown_connections 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 --2- 172.21.15.97:0/1498909184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd7810bcc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 -- 172.21.15.97:0/1498909184 >> 172.21.15.97:0/1498909184 conn(0x7efd78103320 msgr2=0x7efd78105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 -- 172.21.15.97:0/1498909184 shutdown_connections 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.082+0000 7efd7ca00700 1 -- 172.21.15.97:0/1498909184 wait complete. 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.083+0000 7efd7ca00700 1 Processor -- start 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.083+0000 7efd7ca00700 1 -- start start 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd7ca00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd7ca00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd7810d170 con 0x7efd781098d0 2024-04-26T21:20:35.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd75e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd75e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47696/0 (socket says 172.21.15.97:47696) 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd75e00700 1 -- 172.21.15.97:0/872315816 learned_addr learned my addr 172.21.15.97:0/872315816 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd75e00700 1 -- 172.21.15.97:0/872315816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd78109690 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.084+0000 7efd75e00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efd680116a0 tx=0x7efd680046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.085+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd68003b70 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.085+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efd6801a8a0 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.085+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd78107880 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.085+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd68024890 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.085+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd78107dc0 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.086+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7efd68022020 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.086+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd78064a80 con 0x7efd781098d0 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.086+0000 7efd6ea00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 0x7efd5c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:35.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.087+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd6804d530 con 0x7efd781098d0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.087+0000 7efd75400700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 0x7efd5c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.087+0000 7efd75400700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 0x7efd5c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd64006fd0 tx=0x7efd64006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.093+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd6802ad60 con 0x7efd781098d0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.144+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7efd68017ca0 con 0x7efd781098d0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.402+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efd78108040 con 0x7efd781098d0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.402+0000 7efd6ea00700 1 -- 172.21.15.97:0/872315816 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efd6800e260 con 0x7efd781098d0 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.405+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 msgr2=0x7efd5c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.405+0000 7efd7ca00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 0x7efd5c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd64006fd0 tx=0x7efd64006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 msgr2=0x7efd78109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efd680116a0 tx=0x7efd680046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 shutdown_connections 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7efd5c039490 0x7efd5c03b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 --2- 172.21.15.97:0/872315816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd781098d0 0x7efd78109150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:35.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 >> 172.21.15.97:0/872315816 conn(0x7efd78103320 msgr2=0x7efd78103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 shutdown_connections 2024-04-26T21:20:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:34.406+0000 7efd7ca00700 1 -- 172.21.15.97:0/872315816 wait complete. 2024-04-26T21:20:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2024-04-26T21:20:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:35 smithi097 ceph-mon[28127]: mgrmap e4: x(active, since 2s) 2024-04-26T21:20:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/872315816' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:20:36.468 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3778030130' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:20:36.484 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:20:36.485 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.618+0000 7fb7f1e00700 1 Processor -- start 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.618+0000 7fb7f1e00700 1 -- start start 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7f1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7f1e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec108750 con 0x7fb7ec107d60 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47704/0 (socket says 172.21.15.97:47704) 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7eaa00700 1 -- 172.21.15.97:0/327301447 learned_addr learned my addr 172.21.15.97:0/327301447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.619+0000 7fb7eaa00700 1 -- 172.21.15.97:0/327301447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec108f60 con 0x7fb7ec107d60 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.620+0000 7fb7eaa00700 1 --2- 172.21.15.97:0/327301447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098a0 tx=0x7fb7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9cbddd507d8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.620+0000 7fb7e9600700 1 -- 172.21.15.97:0/327301447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7dc004160 con 0x7fb7ec107d60 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.620+0000 7fb7e9600700 1 -- 172.21.15.97:0/327301447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb7dc004300 con 0x7fb7ec107d60 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.620+0000 7fb7e9600700 1 -- 172.21.15.97:0/327301447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7dc0119e0 con 0x7fb7ec107d60 2024-04-26T21:20:36.486 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 -- 172.21.15.97:0/327301447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 msgr2=0x7fb7ec108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/327301447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098a0 tx=0x7fb7dc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 -- 172.21.15.97:0/327301447 shutdown_connections 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/327301447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec108180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 -- 172.21.15.97:0/327301447 >> 172.21.15.97:0/327301447 conn(0x7fb7ec103320 msgr2=0x7fb7ec105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 -- 172.21.15.97:0/327301447 shutdown_connections 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.621+0000 7fb7f1e00700 1 -- 172.21.15.97:0/327301447 wait complete. 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.622+0000 7fb7f1e00700 1 Processor -- start 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.622+0000 7fb7f1e00700 1 -- start start 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.622+0000 7fb7f1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.622+0000 7fb7f1e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec10c190 con 0x7fb7ec107d60 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.623+0000 7fb7eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.623+0000 7fb7eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47714/0 (socket says 172.21.15.97:47714) 2024-04-26T21:20:36.487 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.623+0000 7fb7eaa00700 1 -- 172.21.15.97:0/3778030130 learned_addr learned my addr 172.21.15.97:0/3778030130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.623+0000 7fb7eaa00700 1 -- 172.21.15.97:0/3778030130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec19f4a0 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.623+0000 7fb7eaa00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006b20 tx=0x7fb7dc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.624+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7dc003a60 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.624+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec19f730 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.624+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb7dc003c00 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.624+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7dc024780 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.624+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a2420 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.625+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb7dc022020 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.625+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec064a80 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.625+0000 7fb7f0a00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 0x7fb7d003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.625+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7dc04d6a0 con 0x7fb7ec107d60 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.626+0000 7fb7ea000700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 0x7fb7d003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.626+0000 7fb7ea000700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 0x7fb7d003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:36.488 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.631+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb7dc04d400 con 0x7fb7ec107d60 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.854+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb7ec0008d0 con 0x7fb7ec107d60 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.856+0000 7fb7f0a00700 1 -- 172.21.15.97:0/3778030130 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+461 (secure 0 0 0) 0x7fb7dc01ac50 con 0x7fb7ec107d60 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 msgr2=0x7fb7d003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 0x7fb7d003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 msgr2=0x7fb7ec19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006b20 tx=0x7fb7dc011a60 comp rx=0 tx=0).stop 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 shutdown_connections 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7fb7d00395b0 0x7fb7d003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 --2- 172.21.15.97:0/3778030130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7ec107d60 0x7fb7ec19ef60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 >> 172.21.15.97:0/3778030130 conn(0x7fb7ec103320 msgr2=0x7fb7ec104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 shutdown_connections 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:35.860+0000 7fb7f1e00700 1 -- 172.21.15.97:0/3778030130 wait complete. 2024-04-26T21:20:36.489 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2024-04-26T21:20:37.687 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:37 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/801224705' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T21:20:37.687 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: ignoring --setuser ceph since I am not root 2024-04-26T21:20:37.687 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: ignoring --setgroup ceph since I am not root 2024-04-26T21:20:37.687 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:37.685+0000 7f8250bce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:20:38.066 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.054+0000 7ffbafc00700 1 Processor -- start 2024-04-26T21:20:38.066 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.054+0000 7ffbafc00700 1 -- start start 2024-04-26T21:20:38.066 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.055+0000 7ffbafc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.055+0000 7ffbafc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810ba20 con 0x7ffba810b030 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.055+0000 7ffbad200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.055+0000 7ffbad200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47730/0 (socket says 172.21.15.97:47730) 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.055+0000 7ffbad200700 1 -- 172.21.15.97:0/2228758000 learned_addr learned my addr 172.21.15.97:0/2228758000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.056+0000 7ffbad200700 1 -- 172.21.15.97:0/2228758000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba810c230 con 0x7ffba810b030 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.056+0000 7ffbad200700 1 --2- 172.21.15.97:0/2228758000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db254e096fbadc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.056+0000 7ffba7400700 1 -- 172.21.15.97:0/2228758000 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c004160 con 0x7ffba810b030 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.056+0000 7ffba7400700 1 -- 172.21.15.97:0/2228758000 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffb9c004300 con 0x7ffba810b030 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.056+0000 7ffba7400700 1 -- 172.21.15.97:0/2228758000 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c0119e0 con 0x7ffba810b030 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 -- 172.21.15.97:0/2228758000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 msgr2=0x7ffba810b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 --2- 172.21.15.97:0/2228758000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 -- 172.21.15.97:0/2228758000 shutdown_connections 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 --2- 172.21.15.97:0/2228758000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba810b450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 -- 172.21.15.97:0/2228758000 >> 172.21.15.97:0/2228758000 conn(0x7ffba807e440 msgr2=0x7ffba807e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 -- 172.21.15.97:0/2228758000 shutdown_connections 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.057+0000 7ffbafc00700 1 -- 172.21.15.97:0/2228758000 wait complete. 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.058+0000 7ffbafc00700 1 Processor -- start 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbafc00700 1 -- start start 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbafc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbafc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810f460 con 0x7ffba810b030 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbad200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbad200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47740/0 (socket says 172.21.15.97:47740) 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.059+0000 7ffbad200700 1 -- 172.21.15.97:0/801224705 learned_addr learned my addr 172.21.15.97:0/801224705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffbad200700 1 -- 172.21.15.97:0/801224705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba819f460 con 0x7ffba810b030 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffbad200700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c006b20 tx=0x7ffb9c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c003a60 con 0x7ffba810b030 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffb9c003c00 con 0x7ffba810b030 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c024780 con 0x7ffba810b030 2024-04-26T21:20:38.068 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba819f6f0 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.060+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba81a23e0 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.061+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ffb9c022020 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.062+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba8064a80 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.062+0000 7ffba5600700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 0x7ffb9003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.062+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb9c04d6a0 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.062+0000 7ffba7e00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 0x7ffb9003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.063+0000 7ffba7e00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 0x7ffb9003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006fd0 tx=0x7ffb94006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.068+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffb9c01a950 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.353+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ffba81a25b0 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.407+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ffba81a25b0 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.410+0000 7ffba5600700 1 -- 172.21.15.97:0/801224705 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ffb9c01a430 con 0x7ffba810b030 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 msgr2=0x7ffb9003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 0x7ffb9003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006fd0 tx=0x7ffb94006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 msgr2=0x7ffba819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c006b20 tx=0x7ffb9c011a60 comp rx=0 tx=0).stop 2024-04-26T21:20:38.069 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 shutdown_connections 2024-04-26T21:20:38.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ffb900395b0 0x7ffb9003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:38.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 --2- 172.21.15.97:0/801224705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba810b030 0x7ffba819ef20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:38.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 >> 172.21.15.97:0/801224705 conn(0x7ffba807e440 msgr2=0x7ffba8108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:38.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 shutdown_connections 2024-04-26T21:20:38.070 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:37.411+0000 7ffbafc00700 1 -- 172.21.15.97:0/801224705 wait complete. 2024-04-26T21:20:38.207 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:37.945+0000 7f8250bce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:20:38.207 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:38 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:38.083+0000 7f8250bce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:20:38.496 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:38 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/801224705' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T21:20:38.496 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:38 smithi097 ceph-mon[28127]: mgrmap e5: x(active, since 5s) 2024-04-26T21:20:38.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:38 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:38.494+0000 7f8250bce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:20:38.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:38 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:38.611+0000 7f8250bce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:20:39.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:39 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3106614566' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.659+0000 7ff5b9600700 1 Processor -- start 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.659+0000 7ff5b9600700 1 -- start start 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.660+0000 7ff5b9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.660+0000 7ff5b9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b407ce30 con 0x7ff5b41098d0 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.660+0000 7ff5b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:39.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.660+0000 7ff5b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47772/0 (socket says 172.21.15.97:47772) 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.660+0000 7ff5b2a00700 1 -- 172.21.15.97:0/3758440595 learned_addr learned my addr 172.21.15.97:0/3758440595 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.661+0000 7ff5b2a00700 1 -- 172.21.15.97:0/3758440595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b410c200 con 0x7ff5b41098d0 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.661+0000 7ff5b2a00700 1 --2- 172.21.15.97:0/3758440595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098a0 tx=0x7ff5a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355473f1a1c1a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.661+0000 7ff5b1600700 1 -- 172.21.15.97:0/3758440595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4004160 con 0x7ff5b41098d0 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.661+0000 7ff5b1600700 1 -- 172.21.15.97:0/3758440595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff5a4004300 con 0x7ff5b41098d0 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.661+0000 7ff5b1600700 1 -- 172.21.15.97:0/3758440595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a40119e0 con 0x7ff5b41098d0 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 -- 172.21.15.97:0/3758440595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 msgr2=0x7ff5b410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3758440595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098a0 tx=0x7ff5a4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 -- 172.21.15.97:0/3758440595 shutdown_connections 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3758440595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b410bcc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 -- 172.21.15.97:0/3758440595 >> 172.21.15.97:0/3758440595 conn(0x7ff5b4103320 msgr2=0x7ff5b4105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 -- 172.21.15.97:0/3758440595 shutdown_connections 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.662+0000 7ff5b9600700 1 -- 172.21.15.97:0/3758440595 wait complete. 2024-04-26T21:20:39.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.663+0000 7ff5b9600700 1 Processor -- start 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b9600700 1 -- start start 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b410d170 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47780/0 (socket says 172.21.15.97:47780) 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.664+0000 7ff5b2a00700 1 -- 172.21.15.97:0/3106614566 learned_addr learned my addr 172.21.15.97:0/3106614566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5b2a00700 1 -- 172.21.15.97:0/3106614566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b419f4b0 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5b2a00700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40116a0 tx=0x7ff5a40046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4003b70 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b419f740 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff5a401a8a0 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4024890 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.665+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b41a2430 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff5a4022020 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b4064a80 con 0x7ff5b41098d0 2024-04-26T21:20:39.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5ab400700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 0x7ff59803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5a404d790 con 0x7ff5b41098d0 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5b2000700 1 -- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 msgr2=0x7ff59803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.667+0000 7ff5b2000700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 0x7ff59803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.672+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff5a404d4f0 con 0x7ff5b41098d0 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.868+0000 7ff5b2000700 1 -- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 msgr2=0x7ff59803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.868+0000 7ff5b2000700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 0x7ff59803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.958+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff5b41a2600 con 0x7ff5b41098d0 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.959+0000 7ff5ab400700 1 -- 172.21.15.97:0/3106614566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff5a4046e20 con 0x7ff5b41098d0 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 msgr2=0x7ff59803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 0x7ff59803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 msgr2=0x7ff5b419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:39.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40116a0 tx=0x7ff5a40046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 shutdown_connections 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7ff598039600 0x7ff59803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 --2- 172.21.15.97:0/3106614566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5b41098d0 0x7ff5b419ef70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 >> 172.21.15.97:0/3106614566 conn(0x7ff5b4103320 msgr2=0x7ff5b4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 shutdown_connections 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:38.962+0000 7ff5b9600700 1 -- 172.21.15.97:0/3106614566 wait complete. 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-04-26T21:20:39.643 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2024-04-26T21:20:40.436 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:40 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:40.178+0000 7f8250bce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:20:40.436 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:40 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:40.403+0000 7f8250bce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:20:40.813 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:40 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:40.613+0000 7f8250bce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:20:41.102 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:40 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:40.981+0000 7f8250bce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:20:41.102 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:41 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:41.100+0000 7f8250bce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:20:41.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:41 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:41.350+0000 7f8250bce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:20:42.063 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:41 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:41.601+0000 7f8250bce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:20:42.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:42.398+0000 7f8250bce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:20:43.241 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:42.941+0000 7f8250bce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:20:43.241 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:43.071+0000 7f8250bce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:20:43.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:43.335+0000 7f8250bce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:20:44.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:43.970+0000 7f8250bce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:20:44.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:44.092+0000 7f8250bce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:20:44.712 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:44.409+0000 7f8250bce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:20:44.712 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:44.584+0000 7f8250bce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:20:44.712 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:44.710+0000 7f8250bce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:20:45.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:44.858+0000 7f8250bce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:20:45.563 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:45 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:45.232+0000 7f8250bce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:20:46.259 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:45 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:45.976+0000 7f8250bce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:20:46.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:20:46.259+0000 7f8250bce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:20:46.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: Active manager daemon x restarted 2024-04-26T21:20:46.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: Activating manager daemon x 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: mgrmap e6: x(active, starting, since 0.00468727s) 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:20:46.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:20:46.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:20:46.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: Manager daemon x is now available 2024-04-26T21:20:46.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:46.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:46 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:47.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: Found migration_current of "None". Setting to last migration. 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: mgrmap e7: x(active, since 1.01734s) 2024-04-26T21:20:47.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:47 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.226+0000 7f8c85a00700 1 Processor -- start 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.226+0000 7f8c85a00700 1 -- start start 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c85a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c85a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8007ce30 con 0x7f8c801096c0 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47792/0 (socket says 172.21.15.97:47792) 2024-04-26T21:20:48.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c7fe00700 1 -- 172.21.15.97:0/2975594126 learned_addr learned my addr 172.21.15.97:0/2975594126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.227+0000 7f8c7fe00700 1 -- 172.21.15.97:0/2975594126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8010bff0 con 0x7f8c801096c0 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.228+0000 7f8c7fe00700 1 --2- 172.21.15.97:0/2975594126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c700098a0 tx=0x7f8c70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b02c9c01c0deba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.228+0000 7f8c7ea00700 1 -- 172.21.15.97:0/2975594126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c70004160 con 0x7f8c801096c0 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.228+0000 7f8c7ea00700 1 -- 172.21.15.97:0/2975594126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c70004300 con 0x7f8c801096c0 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.228+0000 7f8c7ea00700 1 -- 172.21.15.97:0/2975594126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c700119e0 con 0x7f8c801096c0 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 -- 172.21.15.97:0/2975594126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 msgr2=0x7f8c8010bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 --2- 172.21.15.97:0/2975594126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c700098a0 tx=0x7f8c70009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 -- 172.21.15.97:0/2975594126 shutdown_connections 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 --2- 172.21.15.97:0/2975594126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8010bab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 -- 172.21.15.97:0/2975594126 >> 172.21.15.97:0/2975594126 conn(0x7f8c80103110 msgr2=0x7f8c80105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 -- 172.21.15.97:0/2975594126 shutdown_connections 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.229+0000 7f8c85a00700 1 -- 172.21.15.97:0/2975594126 wait complete. 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.230+0000 7f8c85a00700 1 Processor -- start 2024-04-26T21:20:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.230+0000 7f8c85a00700 1 -- start start 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.230+0000 7f8c85a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.230+0000 7f8c85a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010cf60 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.231+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.231+0000 7f8c7fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47806/0 (socket says 172.21.15.97:47806) 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.231+0000 7f8c7fe00700 1 -- 172.21.15.97:0/3740540876 learned_addr learned my addr 172.21.15.97:0/3740540876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.231+0000 7f8c7fe00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8019d080 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.231+0000 7f8c7fe00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c700116a0 tx=0x7f8c700046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.232+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c70003b70 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.232+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c7001a8a0 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.232+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c70024890 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.232+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8019d310 con 0x7f8c801096c0 2024-04-26T21:20:48.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.232+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801a0000 con 0x7f8c801096c0 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.233+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8c70022020 con 0x7f8c801096c0 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.233+0000 7f8c7cc00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.234+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c6403c1d0 con 0x7f8c64039600 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.234+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c7004d7b0 con 0x7f8c801096c0 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.234+0000 7f8c7f400700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.234+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.435+0000 7f8c7f400700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.435+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.836+0000 7f8c7f400700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:40.836+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:41.637+0000 7f8c7f400700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:48.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:41.637+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:43.239+0000 7f8c7f400700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2998338527 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:43.239+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:46.267+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f8c7001a430 con 0x7f8c801096c0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:46.268+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 msgr2=0x7f8c6403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:46.268+0000 7f8c7cc00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.273+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f8c7004f6f0 con 0x7f8c801096c0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.274+0000 7f8c7cc00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 0x7f8c6403f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.274+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c6403c1d0 con 0x7f8c6403cef0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.275+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 0x7f8c6403f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.278+0000 7f8c7f400700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 0x7f8c6403f2e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c003a10 tx=0x7f8c6c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.280+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8c6403c1d0 con 0x7f8c6403cef0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.291+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8c80064a80 con 0x7f8c6403cef0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.292+0000 7f8c7cc00700 1 -- 172.21.15.97:0/3740540876 <== mgr.14120 v2:172.21.15.97:6800/1403922799 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8c80064a80 con 0x7f8c6403cef0 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 msgr2=0x7f8c6403f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 0x7f8c6403f2e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c003a10 tx=0x7f8c6c0092b0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 msgr2=0x7f8c8019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c700116a0 tx=0x7f8c700046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 shutdown_connections 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8c6403cef0 0x7f8c6403f2e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c801096c0 0x7f8c8019cb40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 --2- 172.21.15.97:0/3740540876 >> [v2:172.21.15.97:6800/2998338527,v1:172.21.15.97:6801/2998338527] conn(0x7f8c64039600 0x7f8c6403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 >> 172.21.15.97:0/3740540876 conn(0x7f8c80103110 msgr2=0x7f8c80103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 shutdown_connections 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:47.293+0000 7f8c85a00700 1 -- 172.21.15.97:0/3740540876 wait complete. 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2024-04-26T21:20:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: [26/Apr/2024:21:20:47] ENGINE Bus STARTING 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: [26/Apr/2024:21:20:47] ENGINE Serving on https://172.21.15.97:7150 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: [26/Apr/2024:21:20:47] ENGINE Serving on http://172.21.15.97:8765 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: [26/Apr/2024:21:20:47] ENGINE Bus STARTED 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:20:48.487 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:48 smithi097 ceph-mon[28127]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:20:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:49 smithi097 ceph-mon[28127]: mgrmap e8: x(active, since 2s) 2024-04-26T21:20:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:49 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:49 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:49.721 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.789+0000 7f21b5a00700 1 Processor -- start 2024-04-26T21:20:49.721 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.789+0000 7f21b5a00700 1 -- start start 2024-04-26T21:20:49.721 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.790+0000 7f21b5a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.721 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.790+0000 7f21b5a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21b007d7e0 con 0x7f21b007eda0 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.790+0000 7f21aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.790+0000 7f21aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35230/0 (socket says 172.21.15.97:35230) 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.790+0000 7f21aea00700 1 -- 172.21.15.97:0/3365149649 learned_addr learned my addr 172.21.15.97:0/3365149649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.791+0000 7f21aea00700 1 -- 172.21.15.97:0/3365149649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b007d960 con 0x7f21b007eda0 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.791+0000 7f21aea00700 1 --2- 172.21.15.97:0/3365149649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21a00098a0 tx=0x7f21a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680fb3449d479249 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.722 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.791+0000 7f21ad600700 1 -- 172.21.15.97:0/3365149649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a0004160 con 0x7f21b007eda0 2024-04-26T21:20:49.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.791+0000 7f21ad600700 1 -- 172.21.15.97:0/3365149649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f21a0004300 con 0x7f21b007eda0 2024-04-26T21:20:49.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.791+0000 7f21ad600700 1 -- 172.21.15.97:0/3365149649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a00119e0 con 0x7f21b007eda0 2024-04-26T21:20:49.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 -- 172.21.15.97:0/3365149649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 msgr2=0x7f21b007d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 --2- 172.21.15.97:0/3365149649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21a00098a0 tx=0x7f21a0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.723 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 -- 172.21.15.97:0/3365149649 shutdown_connections 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 --2- 172.21.15.97:0/3365149649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b007d2a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 -- 172.21.15.97:0/3365149649 >> 172.21.15.97:0/3365149649 conn(0x7f21b0104490 msgr2=0x7f21b01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 -- 172.21.15.97:0/3365149649 shutdown_connections 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.792+0000 7f21b5a00700 1 -- 172.21.15.97:0/3365149649 wait complete. 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.793+0000 7f21b5a00700 1 Processor -- start 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.793+0000 7f21b5a00700 1 -- start start 2024-04-26T21:20:49.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21b5a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21b5a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010f3a0 con 0x7f21b007eda0 2024-04-26T21:20:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35244/0 (socket says 172.21.15.97:35244) 2024-04-26T21:20:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21aea00700 1 -- 172.21.15.97:0/2600806181 learned_addr learned my addr 172.21.15.97:0/2600806181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.794+0000 7f21aea00700 1 -- 172.21.15.97:0/2600806181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b019f3f0 con 0x7f21b007eda0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21aea00700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21a00116a0 tx=0x7f21a00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a0003b70 con 0x7f21b007eda0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f21a001a8a0 con 0x7f21b007eda0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a0024890 con 0x7f21b007eda0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b019f680 con 0x7f21b007eda0 2024-04-26T21:20:49.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.795+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01a2370 con 0x7f21b007eda0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.796+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f21a0022020 con 0x7f21b007eda0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.796+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b0064a80 con 0x7f21b007eda0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.797+0000 7f21a7400700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 0x7f219403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.797+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f21a004dba0 con 0x7f21b007eda0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.797+0000 7f21ae000700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 0x7f219403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.798+0000 7f21ae000700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 0x7f219403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f219c006fd0 tx=0x7f219c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:48.802+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f21a004d4f0 con 0x7f21b007eda0 2024-04-26T21:20:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.045+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f21b0105690 con 0x7f2194039600 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.056+0000 7f21a7400700 1 -- 172.21.15.97:0/2600806181 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21b0105690 con 0x7f2194039600 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 msgr2=0x7f219403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 0x7f219403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f219c006fd0 tx=0x7f219c006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 msgr2=0x7f21b019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21a00116a0 tx=0x7f21a00046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 shutdown_connections 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2194039600 0x7f219403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 --2- 172.21.15.97:0/2600806181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b007eda0 0x7f21b019eeb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 >> 172.21.15.97:0/2600806181 conn(0x7f21b0104490 msgr2=0x7f21b0104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 shutdown_connections 2024-04-26T21:20:49.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:49.059+0000 7f21b5a00700 1 -- 172.21.15.97:0/2600806181 wait complete. 2024-04-26T21:20:50.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:50 smithi097 ceph-mon[28127]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.325+0000 7fdc16400700 1 Processor -- start 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.325+0000 7fdc16400700 1 -- start start 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc16400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc16400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1007ce10 con 0x7fdc101098f0 2024-04-26T21:20:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc0f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc0f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35246/0 (socket says 172.21.15.97:35246) 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc0f400700 1 -- 172.21.15.97:0/1177997827 learned_addr learned my addr 172.21.15.97:0/1177997827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.326+0000 7fdc0f400700 1 -- 172.21.15.97:0/1177997827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1010c220 con 0x7fdc101098f0 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.327+0000 7fdc0f400700 1 --2- 172.21.15.97:0/1177997827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc000098a0 tx=0x7fdc00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331092733dbc2791 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.327+0000 7fdc0e000700 1 -- 172.21.15.97:0/1177997827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc00004160 con 0x7fdc101098f0 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.327+0000 7fdc0e000700 1 -- 172.21.15.97:0/1177997827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc00004300 con 0x7fdc101098f0 2024-04-26T21:20:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.327+0000 7fdc0e000700 1 -- 172.21.15.97:0/1177997827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc000119e0 con 0x7fdc101098f0 2024-04-26T21:20:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.328+0000 7fdc16400700 1 -- 172.21.15.97:0/1177997827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 msgr2=0x7fdc1010bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.328+0000 7fdc16400700 1 --2- 172.21.15.97:0/1177997827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc000098a0 tx=0x7fdc00009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.328+0000 7fdc16400700 1 -- 172.21.15.97:0/1177997827 shutdown_connections 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.328+0000 7fdc16400700 1 --2- 172.21.15.97:0/1177997827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc1010bce0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.328+0000 7fdc16400700 1 -- 172.21.15.97:0/1177997827 >> 172.21.15.97:0/1177997827 conn(0x7fdc10103360 msgr2=0x7fdc10105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.329+0000 7fdc16400700 1 -- 172.21.15.97:0/1177997827 shutdown_connections 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.329+0000 7fdc16400700 1 -- 172.21.15.97:0/1177997827 wait complete. 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.330+0000 7fdc16400700 1 Processor -- start 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.330+0000 7fdc16400700 1 -- start start 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc16400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc16400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d110 con 0x7fdc101098f0 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc0f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc0f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35254/0 (socket says 172.21.15.97:35254) 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc0f400700 1 -- 172.21.15.97:0/2371194260 learned_addr learned my addr 172.21.15.97:0/2371194260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.331+0000 7fdc0f400700 1 -- 172.21.15.97:0/2371194260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc101096b0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.332+0000 7fdc0f400700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc000116a0 tx=0x7fdc000046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.334+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc00003b70 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.334+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc0001a8a0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.334+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc00024890 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.334+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc101078a0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.334+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc10107de0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.335+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdc00022020 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.335+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc10064a80 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.336+0000 7fdc07e00700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 0x7fdbf403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.336+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc0004dbe0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.336+0000 7fdc0ea00700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 0x7fdbf403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.337+0000 7fdc0ea00700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 0x7fdbf403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006fd0 tx=0x7fdbfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.341+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdc0002d9b0 con 0x7fdc101098f0 2024-04-26T21:20:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.558+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fdc10104540 con 0x7fdbf4039600 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.559+0000 7fdc07e00700 1 -- 172.21.15.97:0/2371194260 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fdc10104540 con 0x7fdbf4039600 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 msgr2=0x7fdbf403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 0x7fdbf403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006fd0 tx=0x7fdbfc006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 msgr2=0x7fdc10109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc000116a0 tx=0x7fdc000046a0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 shutdown_connections 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7fdbf4039600 0x7fdbf403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 --2- 172.21.15.97:0/2371194260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc101098f0 0x7fdc10109170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 >> 172.21.15.97:0/2371194260 conn(0x7fdc10103360 msgr2=0x7fdc10103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 shutdown_connections 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:50.562+0000 7fdc16400700 1 -- 172.21.15.97:0/2371194260 wait complete. 2024-04-26T21:20:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2024-04-26T21:20:51.721 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:51 smithi097 ceph-mon[28127]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: Generating public/private rsa key pair. 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: Your identification has been saved in /tmp/tmp3ecp2yee/key. 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: Your public key has been saved in /tmp/tmp3ecp2yee/key.pub. 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: The key fingerprint is: 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: SHA256:+FO112b/tNnasj6xcypavX3d6nTAxvokhl6e3nMBqkQ ceph-9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: The key's randomart image is: 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: +---[RSA 3072]----+ 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | | 2024-04-26T21:20:52.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | | 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | . | 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | . E . = . | 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | . S . o B +| 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | . o o =.=.| 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | + o * +oB| 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | + =.B**@| 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: | oo++@#*| 2024-04-26T21:20:52.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:20:52 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: +----[SHA256]-----+ 2024-04-26T21:20:52.961 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.783+0000 7f36e2400700 1 Processor -- start 2024-04-26T21:20:52.962 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.783+0000 7f36e2400700 1 -- start start 2024-04-26T21:20:52.962 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.784+0000 7f36e2400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.962 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.784+0000 7f36e2400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc10ba40 con 0x7f36dc10b050 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.784+0000 7f36db400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.784+0000 7f36db400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46786/0 (socket says 172.21.15.97:46786) 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.784+0000 7f36db400700 1 -- 172.21.15.97:0/3336830192 learned_addr learned my addr 172.21.15.97:0/3336830192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.785+0000 7f36db400700 1 -- 172.21.15.97:0/3336830192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc10c250 con 0x7f36dc10b050 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.785+0000 7f36db400700 1 --2- 172.21.15.97:0/3336830192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098a0 tx=0x7f36cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c7f64ffcdc98a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.785+0000 7f36da000700 1 -- 172.21.15.97:0/3336830192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36cc004160 con 0x7f36dc10b050 2024-04-26T21:20:52.963 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36da000700 1 -- 172.21.15.97:0/3336830192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f36cc004300 con 0x7f36dc10b050 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36da000700 1 -- 172.21.15.97:0/3336830192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36cc0119e0 con 0x7f36dc10b050 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36e2400700 1 -- 172.21.15.97:0/3336830192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 msgr2=0x7f36dc10b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36e2400700 1 --2- 172.21.15.97:0/3336830192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098a0 tx=0x7f36cc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36e2400700 1 -- 172.21.15.97:0/3336830192 shutdown_connections 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36e2400700 1 --2- 172.21.15.97:0/3336830192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc10b470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.786+0000 7f36e2400700 1 -- 172.21.15.97:0/3336830192 >> 172.21.15.97:0/3336830192 conn(0x7f36dc07e450 msgr2=0x7f36dc07e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.787+0000 7f36e2400700 1 -- 172.21.15.97:0/3336830192 shutdown_connections 2024-04-26T21:20:52.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.787+0000 7f36e2400700 1 -- 172.21.15.97:0/3336830192 wait complete. 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.787+0000 7f36e2400700 1 Processor -- start 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36e2400700 1 -- start start 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36e2400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36e2400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc10f480 con 0x7f36dc10b050 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36db400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36db400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46794/0 (socket says 172.21.15.97:46794) 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.788+0000 7f36db400700 1 -- 172.21.15.97:0/1247864967 learned_addr learned my addr 172.21.15.97:0/1247864967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36db400700 1 -- 172.21.15.97:0/1247864967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc19f420 con 0x7f36dc10b050 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36db400700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006b20 tx=0x7f36cc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36cc003a60 con 0x7f36dc10b050 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f36cc003c00 con 0x7f36dc10b050 2024-04-26T21:20:52.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36cc024780 con 0x7f36dc10b050 2024-04-26T21:20:52.969 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc19f6b0 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.789+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a23a0 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.790+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36dc064a80 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.791+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f36cc022020 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.791+0000 7f36d3e00700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 0x7f36c003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.792+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f36cc04de40 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.792+0000 7f36daa00700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 0x7f36c003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.792+0000 7f36daa00700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 0x7f36c003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:51.797+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f36cc004300 con 0x7f36dc10b050 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.022+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f36dc108ab0 con 0x7f36c0039600 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.199+0000 7f36d3e00700 1 -- 172.21.15.97:0/1247864967 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36dc108ab0 con 0x7f36c0039600 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 msgr2=0x7f36c003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 0x7f36c003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:52.970 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 msgr2=0x7f36dc19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006b20 tx=0x7f36cc011a60 comp rx=0 tx=0).stop 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 shutdown_connections 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f36c0039600 0x7f36c003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 --2- 172.21.15.97:0/1247864967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36dc10b050 0x7f36dc19eee0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.202+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 >> 172.21.15.97:0/1247864967 conn(0x7f36dc07e450 msgr2=0x7f36dc1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.203+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 shutdown_connections 2024-04-26T21:20:52.971 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:52.203+0000 7f36e2400700 1 -- 172.21.15.97:0/1247864967 wait complete. 2024-04-26T21:20:53.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:53 smithi097 ceph-mon[28127]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:20:53.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:53 smithi097 ceph-mon[28127]: Generating ssh key... 2024-04-26T21:20:53.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:53 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:53.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:53 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:20:54.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.573+0000 7f2d63200700 1 Processor -- start 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.573+0000 7f2d63200700 1 -- start start 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.574+0000 7f2d63200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.574+0000 7f2d63200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c0834d0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.574+0000 7f2d61e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.574+0000 7f2d61e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46804/0 (socket says 172.21.15.97:46804) 2024-04-26T21:20:54.505 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.574+0000 7f2d61e00700 1 -- 172.21.15.97:0/26584494 learned_addr learned my addr 172.21.15.97:0/26584494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.575+0000 7f2d61e00700 1 -- 172.21.15.97:0/26584494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c112580 con 0x7f2d5c10fc50 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.575+0000 7f2d61e00700 1 --2- 172.21.15.97:0/26584494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d500098a0 tx=0x7f2d50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34530d121fb2b62c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.575+0000 7f2d60a00700 1 -- 172.21.15.97:0/26584494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d50004160 con 0x7f2d5c10fc50 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.575+0000 7f2d60a00700 1 -- 172.21.15.97:0/26584494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2d50004300 con 0x7f2d5c10fc50 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.575+0000 7f2d60a00700 1 -- 172.21.15.97:0/26584494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d500119e0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 -- 172.21.15.97:0/26584494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 msgr2=0x7f2d5c112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 --2- 172.21.15.97:0/26584494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d500098a0 tx=0x7f2d50009bb0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 -- 172.21.15.97:0/26584494 shutdown_connections 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 --2- 172.21.15.97:0/26584494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c112040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 -- 172.21.15.97:0/26584494 >> 172.21.15.97:0/26584494 conn(0x7f2d5c07e230 msgr2=0x7f2d5c080670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 -- 172.21.15.97:0/26584494 shutdown_connections 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.576+0000 7f2d63200700 1 -- 172.21.15.97:0/26584494 wait complete. 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.577+0000 7f2d63200700 1 Processor -- start 2024-04-26T21:20:54.509 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.577+0000 7f2d63200700 1 -- start start 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d63200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d63200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1135a0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d61e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d61e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46816/0 (socket says 172.21.15.97:46816) 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d61e00700 1 -- 172.21.15.97:0/2907293489 learned_addr learned my addr 172.21.15.97:0/2907293489 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.578+0000 7f2d61e00700 1 -- 172.21.15.97:0/2907293489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c1a35c0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d61e00700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009130 tx=0x7f2d50011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d50003b20 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2d5001a8a0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d50024810 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c1a3850 con 0x7f2d5c10fc50 2024-04-26T21:20:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.579+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a6540 con 0x7f2d5c10fc50 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.580+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2d50022020 con 0x7f2d5c10fc50 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.580+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c064a80 con 0x7f2d5c10fc50 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.581+0000 7f2d56a00700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 0x7f2d4403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.581+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d5004da50 con 0x7f2d5c10fc50 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.581+0000 7f2d61400700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 0x7f2d4403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.582+0000 7f2d61400700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 0x7f2d4403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c006fd0 tx=0x7f2d4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.586+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2d5000b3e0 con 0x7f2d5c10fc50 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.811+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2d5c07f3f0 con 0x7f2d440395b0 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.812+0000 7f2d56a00700 1 -- 172.21.15.97:0/2907293489 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2d5c07f3f0 con 0x7f2d440395b0 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 msgr2=0x7f2d4403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 0x7f2d4403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c006fd0 tx=0x7f2d4c006e40 comp rx=0 tx=0).stop 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 msgr2=0x7f2d5c1a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009130 tx=0x7f2d50011a60 comp rx=0 tx=0).stop 2024-04-26T21:20:54.511 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 shutdown_connections 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f2d440395b0 0x7f2d4403ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 --2- 172.21.15.97:0/2907293489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5c10fc50 0x7f2d5c1a3080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 >> 172.21.15.97:0/2907293489 conn(0x7f2d5c07e230 msgr2=0x7f2d5c07ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 shutdown_connections 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:53.815+0000 7f2d63200700 1 -- 172.21.15.97:0/2907293489 wait complete. 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:20:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T21:20:54.520 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2024-04-26T21:20:54.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:54 smithi097 ceph-mon[28127]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:20:55.697 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:55 smithi097 ceph-mon[28127]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:20:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:20:57 smithi097 ceph-mon[28127]: Deploying cephadm binary to smithi097 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.109+0000 7f5db9600700 1 Processor -- start 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.109+0000 7f5db9600700 1 -- start start 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4109800 con 0x7f5db4108ea0 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46820/0 (socket says 172.21.15.97:46820) 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db2a00700 1 -- 172.21.15.97:0/1418250632 learned_addr learned my addr 172.21.15.97:0/1418250632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:01.996 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.110+0000 7f5db2a00700 1 -- 172.21.15.97:0/1418250632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4109fb0 con 0x7f5db4108ea0 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.111+0000 7f5db2a00700 1 --2- 172.21.15.97:0/1418250632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5da40098a0 tx=0x7f5da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fa1efd2429b875 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.111+0000 7f5db2000700 1 -- 172.21.15.97:0/1418250632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da4004160 con 0x7f5db4108ea0 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.111+0000 7f5db2000700 1 -- 172.21.15.97:0/1418250632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5da4004300 con 0x7f5db4108ea0 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.111+0000 7f5db2000700 1 -- 172.21.15.97:0/1418250632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da40119e0 con 0x7f5db4108ea0 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 -- 172.21.15.97:0/1418250632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 msgr2=0x7f5db41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 --2- 172.21.15.97:0/1418250632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5da40098a0 tx=0x7f5da4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 -- 172.21.15.97:0/1418250632 shutdown_connections 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 --2- 172.21.15.97:0/1418250632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db41092c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 -- 172.21.15.97:0/1418250632 >> 172.21.15.97:0/1418250632 conn(0x7f5db41044a0 msgr2=0x7f5db41068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:01.997 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 -- 172.21.15.97:0/1418250632 shutdown_connections 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.112+0000 7f5db9600700 1 -- 172.21.15.97:0/1418250632 wait complete. 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.113+0000 7f5db9600700 1 Processor -- start 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.113+0000 7f5db9600700 1 -- start start 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.113+0000 7f5db9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.113+0000 7f5db9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5db410d270 con 0x7f5db4108ea0 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.114+0000 7f5db2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.114+0000 7f5db2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46836/0 (socket says 172.21.15.97:46836) 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.114+0000 7f5db2a00700 1 -- 172.21.15.97:0/3079515095 learned_addr learned my addr 172.21.15.97:0/3079515095 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.114+0000 7f5db2a00700 1 -- 172.21.15.97:0/3079515095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db419f450 con 0x7f5db4108ea0 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.114+0000 7f5db2a00700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5da40116a0 tx=0x7f5da40046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.115+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da4003b70 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.115+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5da401a8a0 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.115+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da4024890 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.115+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db419f6e0 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.115+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41a2380 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.116+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5da4022020 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.116+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4064a80 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.116+0000 7f5dabe00700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 0x7f5d9803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.116+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5da404db00 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.117+0000 7f5daa000700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 0x7f5d9803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.117+0000 7f5daa000700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 0x7f5d9803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00ad30 tx=0x7f5d9c0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.122+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5da402c3a0 con 0x7f5db4108ea0 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:20:55.352+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}) -- 0x7f5db41055a0 con 0x7f5d98039600 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.342+0000 7f5dabe00700 1 -- 172.21.15.97:0/3079515095 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5db41055a0 con 0x7f5d98039600 2024-04-26T21:21:02.000 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 msgr2=0x7f5d9803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 0x7f5d9803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00ad30 tx=0x7f5d9c0093f0 comp rx=0 tx=0).stop 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 msgr2=0x7f5db419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5da40116a0 tx=0x7f5da40046a0 comp rx=0 tx=0).stop 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 shutdown_connections 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f5d98039600 0x7f5d9803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 --2- 172.21.15.97:0/3079515095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db4108ea0 0x7f5db419ef10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 >> 172.21.15.97:0/3079515095 conn(0x7f5db41044a0 msgr2=0x7f5db4104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.345+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 shutdown_connections 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:01.346+0000 7f5db9600700 1 -- 172.21.15.97:0/3079515095 wait complete. 2024-04-26T21:21:02.001 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2024-04-26T21:21:02.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:02 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:02.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:02 smithi097 ceph-mon[28127]: Added host smithi097 2024-04-26T21:21:02.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:02 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:03.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T21:21:03.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.698+0000 7f1aade00700 1 Processor -- start 2024-04-26T21:21:03.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.698+0000 7f1aade00700 1 -- start start 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.699+0000 7f1aade00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.699+0000 7f1aade00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa807cd90 con 0x7f1aa8107760 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.699+0000 7f1aa6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.700+0000 7f1aa6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47272/0 (socket says 172.21.15.97:47272) 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.700+0000 7f1aa6a00700 1 -- 172.21.15.97:0/2261865764 learned_addr learned my addr 172.21.15.97:0/2261865764 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.700+0000 7f1aa6a00700 1 -- 172.21.15.97:0/2261865764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81080c0 con 0x7f1aa8107760 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.701+0000 7f1aa6a00700 1 --2- 172.21.15.97:0/2261865764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a980098a0 tx=0x7f1a98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13da4850740830 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.701+0000 7f1aa5600700 1 -- 172.21.15.97:0/2261865764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a98004160 con 0x7f1aa8107760 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.701+0000 7f1aa5600700 1 -- 172.21.15.97:0/2261865764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1a98004300 con 0x7f1aa8107760 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.701+0000 7f1aa5600700 1 -- 172.21.15.97:0/2261865764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a980119e0 con 0x7f1aa8107760 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 -- 172.21.15.97:0/2261865764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 msgr2=0x7f1aa8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 --2- 172.21.15.97:0/2261865764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a980098a0 tx=0x7f1a98009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 -- 172.21.15.97:0/2261865764 shutdown_connections 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 --2- 172.21.15.97:0/2261865764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa8107b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 -- 172.21.15.97:0/2261865764 >> 172.21.15.97:0/2261865764 conn(0x7f1aa8103320 msgr2=0x7f1aa8105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 -- 172.21.15.97:0/2261865764 shutdown_connections 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.702+0000 7f1aade00700 1 -- 172.21.15.97:0/2261865764 wait complete. 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.703+0000 7f1aade00700 1 Processor -- start 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.703+0000 7f1aade00700 1 -- start start 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.703+0000 7f1aade00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.703+0000 7f1aade00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8109090 con 0x7f1aa8107760 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aa6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aa6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47284/0 (socket says 172.21.15.97:47284) 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aa6a00700 1 -- 172.21.15.97:0/2270913379 learned_addr learned my addr 172.21.15.97:0/2270913379 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aa6a00700 1 -- 172.21.15.97:0/2270913379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa819f450 con 0x7f1aa8107760 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aa6a00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a980116a0 tx=0x7f1a980046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.704+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a98003b70 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.705+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1a9801a8a0 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.705+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa819f6e0 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.705+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a98024890 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.705+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a23d0 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.706+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1a98022020 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.706+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa8064a80 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.706+0000 7f1aaca00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 0x7f1a8c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.706+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a9804dbc0 con 0x7f1aa8107760 2024-04-26T21:21:03.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.706+0000 7f1aa6000700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 0x7f1a8c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.707+0000 7f1aa6000700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 0x7f1a8c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006fd0 tx=0x7f1a94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.712+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a9802c3a0 con 0x7f1aa8107760 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.969+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1aa80008d0 con 0x7f1a8c039600 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.980+0000 7f1aaca00700 1 -- 172.21.15.97:0/2270913379 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1aa80008d0 con 0x7f1a8c039600 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 msgr2=0x7f1a8c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 0x7f1a8c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006fd0 tx=0x7f1a94006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 msgr2=0x7f1aa819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a980116a0 tx=0x7f1a980046a0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 shutdown_connections 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f1a8c039600 0x7f1a8c03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 --2- 172.21.15.97:0/2270913379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa8107760 0x7f1aa819ef10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.983+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 >> 172.21.15.97:0/2270913379 conn(0x7f1aa8103320 msgr2=0x7f1aa8195b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.984+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 shutdown_connections 2024-04-26T21:21:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:02.984+0000 7f1aade00700 1 -- 172.21.15.97:0/2270913379 wait complete. 2024-04-26T21:21:03.845 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2024-04-26T21:21:04.229 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:03 smithi097 ceph-mon[28127]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:04.229 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:03 smithi097 ceph-mon[28127]: Saving service mon spec with placement count:5 2024-04-26T21:21:04.229 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:03 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.998+0000 7f8a83400700 1 Processor -- start 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.998+0000 7f8a83400700 1 -- start start 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.998+0000 7f8a83400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.998+0000 7f8a83400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c0801d0 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.999+0000 7f8a80a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.999+0000 7f8a80a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47300/0 (socket says 172.21.15.97:47300) 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:04.999+0000 7f8a80a00700 1 -- 172.21.15.97:0/2430918994 learned_addr learned my addr 172.21.15.97:0/2430918994 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.000+0000 7f8a80a00700 1 -- 172.21.15.97:0/2430918994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c080a30 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.000+0000 7f8a80a00700 1 --2- 172.21.15.97:0/2430918994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a700098a0 tx=0x7f8a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77a3b15d594c3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.000+0000 7f8a7b400700 1 -- 172.21.15.97:0/2430918994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a70004160 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.000+0000 7f8a7b400700 1 -- 172.21.15.97:0/2430918994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8a70004300 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.000+0000 7f8a7b400700 1 -- 172.21.15.97:0/2430918994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a700119e0 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 -- 172.21.15.97:0/2430918994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 msgr2=0x7f8a7c07fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 --2- 172.21.15.97:0/2430918994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a700098a0 tx=0x7f8a70009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 -- 172.21.15.97:0/2430918994 shutdown_connections 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 --2- 172.21.15.97:0/2430918994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c07fc00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 -- 172.21.15.97:0/2430918994 >> 172.21.15.97:0/2430918994 conn(0x7f8a7c07e440 msgr2=0x7f8a7c07e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 -- 172.21.15.97:0/2430918994 shutdown_connections 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.001+0000 7f8a83400700 1 -- 172.21.15.97:0/2430918994 wait complete. 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.002+0000 7f8a83400700 1 Processor -- start 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a83400700 1 -- start start 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a83400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a83400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c081c70 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a80a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a80a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47304/0 (socket says 172.21.15.97:47304) 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.003+0000 7f8a80a00700 1 -- 172.21.15.97:0/863629433 learned_addr learned my addr 172.21.15.97:0/863629433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a80a00700 1 -- 172.21.15.97:0/863629433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c1a3800 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a80a00700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a700116a0 tx=0x7f8a700046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a70003b70 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8a7001a8a0 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a70024890 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c1a3a90 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.004+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c1a6780 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.006+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8a70022020 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.006+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c064a80 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.006+0000 7f8a79600700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 0x7f8a6403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.006+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a7004dbc0 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.006+0000 7f8a7be00700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 0x7f8a6403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.007+0000 7f8a7be00700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 0x7f8a6403bb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a68006fd0 tx=0x7f8a68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.012+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a7002c3a0 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.133+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a7000be10 con 0x7f8a7c07f7e0 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.260+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8a7c0008d0 con 0x7f8a64039660 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.267+0000 7f8a79600700 1 -- 172.21.15.97:0/863629433 <== mgr.14120 v2:172.21.15.97:6800/1403922799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8a7c0008d0 con 0x7f8a64039660 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 msgr2=0x7f8a6403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 0x7f8a6403bb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a68006fd0 tx=0x7f8a68006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 msgr2=0x7f8a7c1a32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8a700116a0 tx=0x7f8a700046a0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 shutdown_connections 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f8a64039660 0x7f8a6403bb20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 --2- 172.21.15.97:0/863629433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a7c07f7e0 0x7f8a7c1a32c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 >> 172.21.15.97:0/863629433 conn(0x7f8a7c07e440 msgr2=0x7f8a7c199fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 shutdown_connections 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:05.270+0000 7f8a83400700 1 -- 172.21.15.97:0/863629433 wait complete. 2024-04-26T21:21:06.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:06 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:06.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:06 smithi097 ceph-mon[28127]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:06.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:06 smithi097 ceph-mon[28127]: Saving service mgr spec with placement count:2 2024-04-26T21:21:06.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:06 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:06.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:06 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:07 smithi097 ceph-mon[28127]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2434092744' entity='client.admin' 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.564+0000 7f0a7c000700 1 Processor -- start 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.564+0000 7f0a7c000700 1 -- start start 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a7c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a7c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7407cdb0 con 0x7f0a74109950 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a79600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a79600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47312/0 (socket says 172.21.15.97:47312) 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a79600700 1 -- 172.21.15.97:0/2079899787 learned_addr learned my addr 172.21.15.97:0/2079899787 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.565+0000 7f0a79600700 1 -- 172.21.15.97:0/2079899787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7410c280 con 0x7f0a74109950 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.566+0000 7f0a79600700 1 --2- 172.21.15.97:0/2079899787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098a0 tx=0x7f0a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0930d5d4a5c7e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.566+0000 7f0a73e00700 1 -- 172.21.15.97:0/2079899787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68004160 con 0x7f0a74109950 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.566+0000 7f0a73e00700 1 -- 172.21.15.97:0/2079899787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a68004300 con 0x7f0a74109950 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.566+0000 7f0a73e00700 1 -- 172.21.15.97:0/2079899787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68011960 con 0x7f0a74109950 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 -- 172.21.15.97:0/2079899787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 msgr2=0x7f0a7410bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2079899787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098a0 tx=0x7f0a68009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 -- 172.21.15.97:0/2079899787 shutdown_connections 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2079899787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7410bd40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 -- 172.21.15.97:0/2079899787 >> 172.21.15.97:0/2079899787 conn(0x7f0a74103320 msgr2=0x7f0a74105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 -- 172.21.15.97:0/2079899787 shutdown_connections 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.567+0000 7f0a7c000700 1 -- 172.21.15.97:0/2079899787 wait complete. 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.568+0000 7f0a7c000700 1 Processor -- start 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.568+0000 7f0a7c000700 1 -- start start 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a7c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a7c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410d1c0 con 0x7f0a74109950 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a79600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a79600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47326/0 (socket says 172.21.15.97:47326) 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a79600700 1 -- 172.21.15.97:0/2434092744 learned_addr learned my addr 172.21.15.97:0/2434092744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a79600700 1 -- 172.21.15.97:0/2434092744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7419f480 con 0x7f0a74109950 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.569+0000 7f0a79600700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a68011670 tx=0x7f0a68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.570+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a6801f070 con 0x7f0a74109950 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.570+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a68003b20 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.570+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a680195f0 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.570+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7419f710 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.570+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a741a2400 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.571+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0a68022070 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.571+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a74064a80 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.572+0000 7f0a72000700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 0x7f0a5c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.572+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a6804da80 con 0x7f0a74109950 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.572+0000 7f0a78c00700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 0x7f0a5c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.573+0000 7f0a78c00700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 0x7f0a5c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.577+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0a6801baf0 con 0x7f0a74109950 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.810+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0a741a29d0 con 0x7f0a74109950 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.813+0000 7f0a72000700 1 -- 172.21.15.97:0/2434092744 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0a6800e260 con 0x7f0a74109950 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.817+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 msgr2=0x7f0a5c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.817+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 0x7f0a5c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.817+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 msgr2=0x7f0a7419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.817+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a68011670 tx=0x7f0a68011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 shutdown_connections 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f0a5c039600 0x7f0a5c03bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 --2- 172.21.15.97:0/2434092744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a74109950 0x7f0a7419ef40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 >> 172.21.15.97:0/2434092744 conn(0x7f0a74103320 msgr2=0x7f0a74103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 shutdown_connections 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:06.818+0000 7f0a7c000700 1 -- 172.21.15.97:0/2434092744 wait complete. 2024-04-26T21:21:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:09 smithi097 ceph-mon[28127]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:09 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:09 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/763548115' entity='client.admin' 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.759+0000 7eff18600700 1 Processor -- start 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.759+0000 7eff18600700 1 -- start start 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.759+0000 7eff18600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.759+0000 7eff18600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff1007ce30 con 0x7eff101098d0 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.760+0000 7eff15c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.760+0000 7eff15c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47330/0 (socket says 172.21.15.97:47330) 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.760+0000 7eff15c00700 1 -- 172.21.15.97:0/3441822383 learned_addr learned my addr 172.21.15.97:0/3441822383 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.760+0000 7eff15c00700 1 -- 172.21.15.97:0/3441822383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1010c200 con 0x7eff101098d0 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.760+0000 7eff15c00700 1 --2- 172.21.15.97:0/3441822383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff040098a0 tx=0x7eff04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dffcad0f0e5bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.761+0000 7eff0fe00700 1 -- 172.21.15.97:0/3441822383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff04004160 con 0x7eff101098d0 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.761+0000 7eff0fe00700 1 -- 172.21.15.97:0/3441822383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff04004300 con 0x7eff101098d0 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.761+0000 7eff0fe00700 1 -- 172.21.15.97:0/3441822383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff04011960 con 0x7eff101098d0 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 -- 172.21.15.97:0/3441822383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 msgr2=0x7eff1010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 --2- 172.21.15.97:0/3441822383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff040098a0 tx=0x7eff04009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 -- 172.21.15.97:0/3441822383 shutdown_connections 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 --2- 172.21.15.97:0/3441822383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff1010bcc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 -- 172.21.15.97:0/3441822383 >> 172.21.15.97:0/3441822383 conn(0x7eff10103320 msgr2=0x7eff10105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 -- 172.21.15.97:0/3441822383 shutdown_connections 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.762+0000 7eff18600700 1 -- 172.21.15.97:0/3441822383 wait complete. 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.763+0000 7eff18600700 1 Processor -- start 2024-04-26T21:21:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff18600700 1 -- start start 2024-04-26T21:21:09.730 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff18600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.730 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff18600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff1010d1b0 con 0x7eff101098d0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff15c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff15c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47344/0 (socket says 172.21.15.97:47344) 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.764+0000 7eff15c00700 1 -- 172.21.15.97:0/763548115 learned_addr learned my addr 172.21.15.97:0/763548115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.765+0000 7eff15c00700 1 -- 172.21.15.97:0/763548115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff10109690 con 0x7eff101098d0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.765+0000 7eff15c00700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff04004660 tx=0x7eff04011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.765+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff04003aa0 con 0x7eff101098d0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.766+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff0401b8d0 con 0x7eff101098d0 2024-04-26T21:21:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.766+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff04023820 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.766+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff10107880 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.766+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff10107dc0 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.767+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7eff0401b440 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.767+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff10064a80 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.767+0000 7eff0e000700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 0x7efef803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.767+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff0404df30 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.768+0000 7eff15200700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 0x7efef803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.768+0000 7eff15200700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 0x7efef803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efefc006fd0 tx=0x7efefc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.773+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff0401f030 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:08.997+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7eff10108040 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.005+0000 7eff0e000700 1 -- 172.21.15.97:0/763548115 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7eff0401a3d0 con 0x7eff101098d0 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.008+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 msgr2=0x7efef803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.008+0000 7eff18600700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 0x7efef803ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efefc006fd0 tx=0x7efefc006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:09.732 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.008+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 msgr2=0x7eff10109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.008+0000 7eff18600700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff04004660 tx=0x7eff04011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 shutdown_connections 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7efef8039560 0x7efef803ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 --2- 172.21.15.97:0/763548115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff101098d0 0x7eff10109150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 >> 172.21.15.97:0/763548115 conn(0x7eff10103320 msgr2=0x7eff10103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 shutdown_connections 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:09.009+0000 7eff18600700 1 -- 172.21.15.97:0/763548115 wait complete. 2024-04-26T21:21:09.733 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2024-04-26T21:21:11.692 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-mon[28127]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:11.692 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-mon[28127]: from='mgr.14120 172.21.15.97:0/4201561102' entity='mgr.x' 2024-04-26T21:21:11.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/82860277' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T21:21:11.986 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: ignoring --setuser ceph since I am not root 2024-04-26T21:21:11.987 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: ignoring --setgroup ceph since I am not root 2024-04-26T21:21:11.987 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:11 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:11.873+0000 7f3b19a7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:21:12.254 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:12.120+0000 7f3b19a7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:21:12.254 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:12.252+0000 7f3b19a7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.352+0000 7f68b5400700 1 Processor -- start 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.352+0000 7f68b5400700 1 -- start start 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.352+0000 7f68b5400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.353+0000 7f68b5400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68b007d7c0 con 0x7f68b007ed80 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.353+0000 7f68ae800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.353+0000 7f68ae800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47352/0 (socket says 172.21.15.97:47352) 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.353+0000 7f68ae800700 1 -- 172.21.15.97:0/447921543 learned_addr learned my addr 172.21.15.97:0/447921543 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.354+0000 7f68ae800700 1 -- 172.21.15.97:0/447921543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b007d940 con 0x7f68b007ed80 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.354+0000 7f68ae800700 1 --2- 172.21.15.97:0/447921543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098a0 tx=0x7f68a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4635455d8e1dd9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.354+0000 7f68ad400700 1 -- 172.21.15.97:0/447921543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68a0004160 con 0x7f68b007ed80 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.354+0000 7f68ad400700 1 -- 172.21.15.97:0/447921543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a0004300 con 0x7f68b007ed80 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.354+0000 7f68ad400700 1 -- 172.21.15.97:0/447921543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68a0011960 con 0x7f68b007ed80 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 -- 172.21.15.97:0/447921543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 msgr2=0x7f68b007d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 --2- 172.21.15.97:0/447921543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098a0 tx=0x7f68a0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 -- 172.21.15.97:0/447921543 shutdown_connections 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 --2- 172.21.15.97:0/447921543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b007d280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 -- 172.21.15.97:0/447921543 >> 172.21.15.97:0/447921543 conn(0x7f68b0104490 msgr2=0x7f68b01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 -- 172.21.15.97:0/447921543 shutdown_connections 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.355+0000 7f68b5400700 1 -- 172.21.15.97:0/447921543 wait complete. 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.356+0000 7f68b5400700 1 Processor -- start 2024-04-26T21:21:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68b5400700 1 -- start start 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68b5400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68b5400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68b010f3b0 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68ae800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68ae800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47362/0 (socket says 172.21.15.97:47362) 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68ae800700 1 -- 172.21.15.97:0/82860277 learned_addr learned my addr 172.21.15.97:0/82860277 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.357+0000 7f68ae800700 1 -- 172.21.15.97:0/82860277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b019f410 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68ae800700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68a0011670 tx=0x7f68a0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68a001f070 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a0003b20 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68a00195f0 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b019f6a0 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.358+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a2390 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.359+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f68a0022070 con 0x7f68b007ed80 2024-04-26T21:21:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.360+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b0064a80 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.360+0000 7f68a7400700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 0x7f689403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.360+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f68a004db70 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.360+0000 7f68ade00700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 0x7f689403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.361+0000 7f68ade00700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 0x7f689403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f689c006fd0 tx=0x7f689c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.366+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68a004d4c0 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:10.658+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f68b01a2940 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.555+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f68a001baf0 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.560+0000 7f68a7400700 1 -- 172.21.15.97:0/82860277 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f68a000e260 con 0x7f68b007ed80 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.561+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 msgr2=0x7f689403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.561+0000 7f68b5400700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 0x7f689403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f689c006fd0 tx=0x7f689c006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 msgr2=0x7f68b019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68a0011670 tx=0x7f68a0011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 shutdown_connections 2024-04-26T21:21:12.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f68940395b0 0x7f689403ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 --2- 172.21.15.97:0/82860277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b007ed80 0x7f68b019eed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 >> 172.21.15.97:0/82860277 conn(0x7f68b0104490 msgr2=0x7f68b0104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 shutdown_connections 2024-04-26T21:21:12.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:11.562+0000 7f68b5400700 1 -- 172.21.15.97:0/82860277 wait complete. 2024-04-26T21:21:12.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/82860277' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T21:21:12.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-mon[28127]: mgrmap e9: x(active, since 25s) 2024-04-26T21:21:12.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:12.657+0000 7f3b19a7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:21:12.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:12.771+0000 7f3b19a7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:21:14.267 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:13 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1115955330' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:21:14.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:14.342+0000 7f3b19a7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:21:14.602 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:21:14.602 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T21:21:14.602 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.635+0000 7f16bf800700 1 Processor -- start 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.635+0000 7f16bf800700 1 -- start start 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.636+0000 7f16bf800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.636+0000 7f16bf800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8109830 con 0x7f16b8108ed0 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.636+0000 7f16bce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.636+0000 7f16bce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52876/0 (socket says 172.21.15.97:52876) 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.636+0000 7f16bce00700 1 -- 172.21.15.97:0/3082589502 learned_addr learned my addr 172.21.15.97:0/3082589502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.637+0000 7f16bce00700 1 -- 172.21.15.97:0/3082589502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b8109f90 con 0x7f16b8108ed0 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.637+0000 7f16bce00700 1 --2- 172.21.15.97:0/3082589502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0098a0 tx=0x7f16ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fef7da7c8ade2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.603 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.637+0000 7f16b7400700 1 -- 172.21.15.97:0/3082589502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac004160 con 0x7f16b8108ed0 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.637+0000 7f16b7400700 1 -- 172.21.15.97:0/3082589502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ac004300 con 0x7f16b8108ed0 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.637+0000 7f16b7400700 1 -- 172.21.15.97:0/3082589502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac011960 con 0x7f16b8108ed0 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 -- 172.21.15.97:0/3082589502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 msgr2=0x7f16b81092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 --2- 172.21.15.97:0/3082589502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0098a0 tx=0x7f16ac009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 -- 172.21.15.97:0/3082589502 shutdown_connections 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 --2- 172.21.15.97:0/3082589502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b81092f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 -- 172.21.15.97:0/3082589502 >> 172.21.15.97:0/3082589502 conn(0x7f16b8104490 msgr2=0x7f16b81068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 -- 172.21.15.97:0/3082589502 shutdown_connections 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.638+0000 7f16bf800700 1 -- 172.21.15.97:0/3082589502 wait complete. 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.639+0000 7f16bf800700 1 Processor -- start 2024-04-26T21:21:14.604 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.639+0000 7f16bf800700 1 -- start start 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bf800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bf800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f16b810d250 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52886/0 (socket says 172.21.15.97:52886) 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bce00700 1 -- 172.21.15.97:0/1115955330 learned_addr learned my addr 172.21.15.97:0/1115955330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.640+0000 7f16bce00700 1 -- 172.21.15.97:0/1115955330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b807f030 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16bce00700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16ac011670 tx=0x7f16ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac01f070 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ac003b20 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b807d220 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac0195f0 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.641+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b807d760 con 0x7f16b8108ed0 2024-04-26T21:21:14.605 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.642+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b8064a80 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.643+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f16ac022070 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.643+0000 7f16b5600700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 0x7f16a003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.643+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16ac04dd10 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.643+0000 7f16b7e00700 1 -- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 msgr2=0x7f16a003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.643+0000 7f16b7e00700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 0x7f16a003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.649+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16ac04d660 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.844+0000 7f16b7e00700 1 -- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 msgr2=0x7f16a003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.844+0000 7f16b7e00700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 0x7f16a003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.925+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f16b807dd50 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.926+0000 7f16b5600700 1 -- 172.21.15.97:0/1115955330 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f16ac019cf0 con 0x7f16b8108ed0 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.928+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 msgr2=0x7f16a003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.928+0000 7f16bf800700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 0x7f16a003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.606 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.928+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 msgr2=0x7f16b807eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.928+0000 7f16bf800700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16ac011670 tx=0x7f16ac011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 shutdown_connections 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f16a0039660 0x7f16a003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 --2- 172.21.15.97:0/1115955330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16b8108ed0 0x7f16b807eaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 >> 172.21.15.97:0/1115955330 conn(0x7f16b8104490 msgr2=0x7f16b8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 shutdown_connections 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:13.929+0000 7f16bf800700 1 -- 172.21.15.97:0/1115955330 wait complete. 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-04-26T21:21:14.607 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2024-04-26T21:21:14.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:14.570+0000 7f3b19a7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:21:14.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:14.794+0000 7f3b19a7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:21:15.242 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:15.192+0000 7f3b19a7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:21:15.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:15.318+0000 7f3b19a7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:21:15.824 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:15.572+0000 7f3b19a7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:21:15.824 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:15.822+0000 7f3b19a7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:21:17.063 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:16 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:16.604+0000 7f3b19a7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:21:17.539 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:17 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:17.145+0000 7f3b19a7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:21:17.539 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:17 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:17.273+0000 7f3b19a7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:21:17.539 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:17 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:17.537+0000 7f3b19a7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:21:18.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:18.179+0000 7f3b19a7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:21:18.564 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:18.303+0000 7f3b19a7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:21:18.923 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:18.620+0000 7f3b19a7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:21:18.923 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:18.796+0000 7f3b19a7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:21:18.923 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:18 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:18.921+0000 7f3b19a7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:21:19.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:19 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:19.070+0000 7f3b19a7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:21:19.814 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:19 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:19.437+0000 7f3b19a7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:21:20.480 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:20.184+0000 7f3b19a7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:21:20.813 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:21:20.478+0000 7f3b19a7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: Active manager daemon x restarted 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: Activating manager daemon x 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: mgrmap e10: x(active, starting, since 0.00659404s) 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:21:20.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:20 smithi097 ceph-mon[28127]: Manager daemon x is now available 2024-04-26T21:21:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:21:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:21:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:21 smithi097 ceph-mon[28127]: mgrmap e11: x(active, since 1.01748s) 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.240+0000 7f6a5be00700 1 Processor -- start 2024-04-26T21:21:22.175 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5be00700 1 -- start start 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5be00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c07d810 con 0x7f6a5c07ede0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52894/0 (socket says 172.21.15.97:52894) 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.241+0000 7f6a5aa00700 1 -- 172.21.15.97:0/1836505259 learned_addr learned my addr 172.21.15.97:0/1836505259 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.242+0000 7f6a5aa00700 1 -- 172.21.15.97:0/1836505259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c07d990 con 0x7f6a5c07ede0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.242+0000 7f6a5aa00700 1 --2- 172.21.15.97:0/1836505259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0098a0 tx=0x7f6a4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9591b648cc687111 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.242+0000 7f6a59600700 1 -- 172.21.15.97:0/1836505259 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c004160 con 0x7f6a5c07ede0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a59600700 1 -- 172.21.15.97:0/1836505259 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a4c004300 con 0x7f6a5c07ede0 2024-04-26T21:21:22.177 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a59600700 1 -- 172.21.15.97:0/1836505259 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c011960 con 0x7f6a5c07ede0 2024-04-26T21:21:22.178 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a5be00700 1 -- 172.21.15.97:0/1836505259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 msgr2=0x7f6a5c07d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a5be00700 1 --2- 172.21.15.97:0/1836505259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0098a0 tx=0x7f6a4c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a5be00700 1 -- 172.21.15.97:0/1836505259 shutdown_connections 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a5be00700 1 --2- 172.21.15.97:0/1836505259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c07d240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.243+0000 7f6a5be00700 1 -- 172.21.15.97:0/1836505259 >> 172.21.15.97:0/1836505259 conn(0x7f6a5c1041c0 msgr2=0x7f6a5c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.244+0000 7f6a5be00700 1 -- 172.21.15.97:0/1836505259 shutdown_connections 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.244+0000 7f6a5be00700 1 -- 172.21.15.97:0/1836505259 wait complete. 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.244+0000 7f6a5be00700 1 Processor -- start 2024-04-26T21:21:22.180 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5be00700 1 -- start start 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5be00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10f1c0 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52908/0 (socket says 172.21.15.97:52908) 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.245+0000 7f6a5aa00700 1 -- 172.21.15.97:0/2971075247 learned_addr learned my addr 172.21.15.97:0/2971075247 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a5aa00700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c19f250 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a5aa00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011670 tx=0x7f6a4c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c01f070 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a4c003b20 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c19f4e0 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c0195f0 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.246+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1a21c0 con 0x7f6a5c07ede0 2024-04-26T21:21:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6a4c022070 con 0x7f6a5c07ede0 2024-04-26T21:21:22.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a53400700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a4003c230 con 0x7f6a40039660 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a4c04dad0 con 0x7f6a5c07ede0 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a5a000700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.248+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.449+0000 7f6a5a000700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.449+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.850+0000 7f6a5a000700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:15.850+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:16.651+0000 7f6a5a000700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:16.651+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:18.253+0000 7f6a5a000700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1403922799 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:18.253+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:20.487+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6a4c01b490 con 0x7f6a5c07ede0 2024-04-26T21:21:22.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:20.487+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 msgr2=0x7f6a4003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:20.487+0000 7f6a53400700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.491+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6a4c02cd00 con 0x7f6a5c07ede0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.491+0000 7f6a53400700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 0x7f6a4003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.492+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a4003c230 con 0x7f6a4003cff0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.493+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 0x7f6a4003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.494+0000 7f6a5a000700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 0x7f6a4003f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6a48003a10 tx=0x7f6a48009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.495+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6a4003c230 con 0x7f6a4003cff0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.505+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6a5c064a80 con 0x7f6a4003cff0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.506+0000 7f6a53400700 1 -- 172.21.15.97:0/2971075247 <== mgr.14152 v2:172.21.15.97:6800/3172837618 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6a5c064a80 con 0x7f6a4003cff0 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 msgr2=0x7f6a4003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 0x7f6a4003f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6a48003a10 tx=0x7f6a48009270 comp rx=0 tx=0).stop 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 msgr2=0x7f6a5c19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011670 tx=0x7f6a4c011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 shutdown_connections 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6a4003cff0 0x7f6a4003f3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a5c07ede0 0x7f6a5c19ed10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 --2- 172.21.15.97:0/2971075247 >> [v2:172.21.15.97:6800/1403922799,v1:172.21.15.97:6801/1403922799] conn(0x7f6a40039660 0x7f6a4003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 >> 172.21.15.97:0/2971075247 conn(0x7f6a5c1041c0 msgr2=0x7f6a5c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 shutdown_connections 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:21.507+0000 7f6a5be00700 1 -- 172.21.15.97:0/2971075247 wait complete. 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2024-04-26T21:21:22.188 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: [26/Apr/2024:21:21:21] ENGINE Bus STARTING 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: [26/Apr/2024:21:21:21] ENGINE Serving on https://172.21.15.97:7150 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: [26/Apr/2024:21:21:21] ENGINE Serving on http://172.21.15.97:8765 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: [26/Apr/2024:21:21:21] ENGINE Bus STARTED 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:21:22.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:22 smithi097 ceph-mon[28127]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:21:23.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:23 smithi097 ceph-mon[28127]: mgrmap e12: x(active, since 2s) 2024-04-26T21:21:23.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:23.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:24.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.931+0000 7fbba0a00700 1 Processor -- start 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.931+0000 7fbba0a00700 1 -- start start 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.931+0000 7fbba0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.931+0000 7fbba0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1097e0 con 0x7fbb9c108e80 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.932+0000 7fbb99e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.932+0000 7fbb99e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57484/0 (socket says 172.21.15.97:57484) 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.932+0000 7fbb99e00700 1 -- 172.21.15.97:0/1496280492 learned_addr learned my addr 172.21.15.97:0/1496280492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.932+0000 7fbb99e00700 1 -- 172.21.15.97:0/1496280492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c109f90 con 0x7fbb9c108e80 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.932+0000 7fbb99e00700 1 --2- 172.21.15.97:0/1496280492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0098a0 tx=0x7fbb8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9acc3ae743a42f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbb98a00700 1 -- 172.21.15.97:0/1496280492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8c004160 con 0x7fbb9c108e80 2024-04-26T21:21:24.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbb98a00700 1 -- 172.21.15.97:0/1496280492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb8c004300 con 0x7fbb9c108e80 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbb98a00700 1 -- 172.21.15.97:0/1496280492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8c011960 con 0x7fbb9c108e80 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 -- 172.21.15.97:0/1496280492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 msgr2=0x7fbb9c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 --2- 172.21.15.97:0/1496280492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0098a0 tx=0x7fbb8c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 -- 172.21.15.97:0/1496280492 shutdown_connections 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 --2- 172.21.15.97:0/1496280492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c1092a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 -- 172.21.15.97:0/1496280492 >> 172.21.15.97:0/1496280492 conn(0x7fbb9c104480 msgr2=0x7fbb9c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.933+0000 7fbba0a00700 1 -- 172.21.15.97:0/1496280492 shutdown_connections 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.934+0000 7fbba0a00700 1 -- 172.21.15.97:0/1496280492 wait complete. 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbba0a00700 1 Processor -- start 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbba0a00700 1 -- start start 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbba0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbba0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d250 con 0x7fbb9c108e80 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbb99e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbb99e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57486/0 (socket says 172.21.15.97:57486) 2024-04-26T21:21:24.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.935+0000 7fbb99e00700 1 -- 172.21.15.97:0/4118278807 learned_addr learned my addr 172.21.15.97:0/4118278807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbb99e00700 1 -- 172.21.15.97:0/4118278807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c07f030 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbb99e00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c011670 tx=0x7fbb8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8c01f070 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb8c003b20 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8c0195f0 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.936+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c07d220 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.937+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c07d760 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.938+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fbb8c022070 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.938+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c10d3d0 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.938+0000 7fbb92a00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 0x7fbb8003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.938+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb8c04dc60 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.938+0000 7fbb99400700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 0x7fbb8003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.939+0000 7fbb99400700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 0x7fbb8003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb88006fd0 tx=0x7fbb88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:22.944+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb8c01b490 con 0x7fbb9c108e80 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.179+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbb9c07e090 con 0x7fbb80039660 2024-04-26T21:21:24.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.336+0000 7fbb92a00700 1 -- 172.21.15.97:0/4118278807 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbb9c07e090 con 0x7fbb80039660 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 msgr2=0x7fbb8003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 0x7fbb8003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb88006fd0 tx=0x7fbb88006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 msgr2=0x7fbb9c07eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c011670 tx=0x7fbb8c011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 shutdown_connections 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb80039660 0x7fbb8003bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 --2- 172.21.15.97:0/4118278807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb9c108e80 0x7fbb9c07eaf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 >> 172.21.15.97:0/4118278807 conn(0x7fbb9c104480 msgr2=0x7fbb9c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 shutdown_connections 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:23.340+0000 7fbba0a00700 1 -- 172.21.15.97:0/4118278807 wait complete. 2024-04-26T21:21:24.025 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2024-04-26T21:21:24.885 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:24 smithi097 ceph-mon[28127]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:24.885 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:25.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3ucp/5r7kf6EZS3bNhQEkOmGUb0Klh7pEuKvoZh3jZxCwcP2D0buK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714166485, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.623+0000 7fef66400700 1 Processor -- start 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.623+0000 7fef66400700 1 -- start start 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.623+0000 7fef66400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.624+0000 7fef66400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef6007d7a0 con 0x7fef6007ed60 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.624+0000 7fef5f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.624+0000 7fef5f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57498/0 (socket says 172.21.15.97:57498) 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.624+0000 7fef5f400700 1 -- 172.21.15.97:0/2098491216 learned_addr learned my addr 172.21.15.97:0/2098491216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.625+0000 7fef5f400700 1 -- 172.21.15.97:0/2098491216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6007d920 con 0x7fef6007ed60 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.625+0000 7fef5f400700 1 --2- 172.21.15.97:0/2098491216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef500098a0 tx=0x7fef50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48dbeaed98fc40b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.625+0000 7fef5e000700 1 -- 172.21.15.97:0/2098491216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef50004160 con 0x7fef6007ed60 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.625+0000 7fef5e000700 1 -- 172.21.15.97:0/2098491216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef50004300 con 0x7fef6007ed60 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.625+0000 7fef5e000700 1 -- 172.21.15.97:0/2098491216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef50011960 con 0x7fef6007ed60 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 -- 172.21.15.97:0/2098491216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 msgr2=0x7fef6007d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 --2- 172.21.15.97:0/2098491216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef500098a0 tx=0x7fef50009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 -- 172.21.15.97:0/2098491216 shutdown_connections 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 --2- 172.21.15.97:0/2098491216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6007d260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 -- 172.21.15.97:0/2098491216 >> 172.21.15.97:0/2098491216 conn(0x7fef60104490 msgr2=0x7fef601068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 -- 172.21.15.97:0/2098491216 shutdown_connections 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.626+0000 7fef66400700 1 -- 172.21.15.97:0/2098491216 wait complete. 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.627+0000 7fef66400700 1 Processor -- start 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.627+0000 7fef66400700 1 -- start start 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef66400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef66400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef6010f370 con 0x7fef6007ed60 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef5f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef5f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57512/0 (socket says 172.21.15.97:57512) 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef5f400700 1 -- 172.21.15.97:0/1789012460 learned_addr learned my addr 172.21.15.97:0/1789012460 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.628+0000 7fef5f400700 1 -- 172.21.15.97:0/1789012460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6019f430 con 0x7fef6007ed60 2024-04-26T21:21:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef5f400700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef50011670 tx=0x7fef50011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef5001f070 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef50003b20 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef66400700 1 -- 172.21.15.97:0/1789012460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6019f6c0 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef500195f0 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.629+0000 7fef66400700 1 -- 172.21.15.97:0/1789012460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601a23b0 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.631+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef60064a80 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.631+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fef50022070 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.631+0000 7fef57e00700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 0x7fef4403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.631+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef5004dbd0 con 0x7fef6007ed60 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.632+0000 7fef5ea00700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 0x7fef4403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.632+0000 7fef5ea00700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 0x7fef4403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006fd0 tx=0x7fef4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:25.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.637+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fef5001b440 con 0x7fef6007ed60 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:24.884+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fef600641c0 con 0x7fef44039660 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.126+0000 7fef57e00700 1 -- 172.21.15.97:0/1789012460 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fef600641c0 con 0x7fef44039660 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 msgr2=0x7fef4403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 0x7fef4403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006fd0 tx=0x7fef4c006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 msgr2=0x7fef6019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef50011670 tx=0x7fef50011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 shutdown_connections 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fef44039660 0x7fef4403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 --2- 172.21.15.97:0/1789012460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6007ed60 0x7fef6019eef0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.129+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 >> 172.21.15.97:0/1789012460 conn(0x7fef60104490 msgr2=0x7fef60104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.130+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 shutdown_connections 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:25.130+0000 7fef55600700 1 -- 172.21.15.97:0/1789012460 wait complete. 2024-04-26T21:21:25.808 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2024-04-26T21:21:26.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:26 smithi097 ceph-mon[28127]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:26.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:26.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:27.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:27 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2475629760' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.393+0000 7f56c1c00700 1 Processor -- start 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56c1c00700 1 -- start start 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56c1c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56c1c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc108750 con 0x7f56bc107d60 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56baa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56baa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57524/0 (socket says 172.21.15.97:57524) 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.394+0000 7f56baa00700 1 -- 172.21.15.97:0/3775685667 learned_addr learned my addr 172.21.15.97:0/3775685667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.395+0000 7f56baa00700 1 -- 172.21.15.97:0/3775685667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc108f60 con 0x7f56bc107d60 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.395+0000 7f56baa00700 1 --2- 172.21.15.97:0/3775685667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0098a0 tx=0x7f56ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d2ef10c95196a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.395+0000 7f56b9600700 1 -- 172.21.15.97:0/3775685667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac004160 con 0x7f56bc107d60 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.395+0000 7f56b9600700 1 -- 172.21.15.97:0/3775685667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac004300 con 0x7f56bc107d60 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.395+0000 7f56b9600700 1 -- 172.21.15.97:0/3775685667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac011960 con 0x7f56bc107d60 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 -- 172.21.15.97:0/3775685667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 msgr2=0x7f56bc108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 --2- 172.21.15.97:0/3775685667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0098a0 tx=0x7f56ac009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 -- 172.21.15.97:0/3775685667 shutdown_connections 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 --2- 172.21.15.97:0/3775685667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc108180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 -- 172.21.15.97:0/3775685667 >> 172.21.15.97:0/3775685667 conn(0x7f56bc103320 msgr2=0x7f56bc105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.396+0000 7f56c1c00700 1 -- 172.21.15.97:0/3775685667 shutdown_connections 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.397+0000 7f56c1c00700 1 -- 172.21.15.97:0/3775685667 wait complete. 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.397+0000 7f56c1c00700 1 Processor -- start 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56c1c00700 1 -- start start 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56c1c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56c1c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc10c190 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56baa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56baa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57538/0 (socket says 172.21.15.97:57538) 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.398+0000 7f56baa00700 1 -- 172.21.15.97:0/2475629760 learned_addr learned my addr 172.21.15.97:0/2475629760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56baa00700 1 -- 172.21.15.97:0/2475629760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc19d2e0 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56baa00700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56ac011670 tx=0x7f56ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac003aa0 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc19d570 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac01b8d0 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.399+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac0197f0 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.400+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a0260 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.401+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f56ac01b440 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.401+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc10c310 con 0x7f56bc107d60 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.401+0000 7f56b3400700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 0x7f56a003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.401+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56ac04e080 con 0x7f56bc107d60 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.402+0000 7f56ba000700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 0x7f56a003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.402+0000 7f56ba000700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 0x7f56a003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.407+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f56ac01f030 con 0x7f56bc107d60 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.628+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f56bc02e6d0 con 0x7f56bc107d60 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.629+0000 7f56b3400700 1 -- 172.21.15.97:0/2475629760 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f56ac010380 con 0x7f56bc107d60 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.632+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 msgr2=0x7f56a003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.632+0000 7f56c1c00700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 0x7f56a003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.632+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 msgr2=0x7f56bc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.632+0000 7f56c1c00700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56ac011670 tx=0x7f56ac011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 shutdown_connections 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f56a00395b0 0x7f56a003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 --2- 172.21.15.97:0/2475629760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56bc107d60 0x7f56bc19cda0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 >> 172.21.15.97:0/2475629760 conn(0x7f56bc103320 msgr2=0x7f56bc1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:28.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 shutdown_connections 2024-04-26T21:21:28.254 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:26.633+0000 7f56c1c00700 1 -- 172.21.15.97:0/2475629760 wait complete. 2024-04-26T21:21:28.283 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:21:28.283 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-26T21:21:28.305 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:21:28.306 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-26T21:21:28.306 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-26T21:21:28.306 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097.front.sepia.ceph.com:8443/ 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout: Password: lmmsyo9kdg 2024-04-26T21:21:28.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:28.314 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config directory 2024-04-26T21:21:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:21:28.461 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:28 smithi097 ceph-mon[28127]: mgrmap e13: x(active, since 6s) 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.922+0000 7fc5b0a00700 1 Processor -- start 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.922+0000 7fc5b0a00700 1 -- start start 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.923+0000 7fc5b0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.923+0000 7fc5b0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac07d830 con 0x7fc5ac07ee00 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.923+0000 7fc5a9e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.923+0000 7fc5a9e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57548/0 (socket says 172.21.15.97:57548) 2024-04-26T21:21:29.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.923+0000 7fc5a9e00700 1 -- 172.21.15.97:0/2388564661 learned_addr learned my addr 172.21.15.97:0/2388564661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:29.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.924+0000 7fc5a9e00700 1 -- 172.21.15.97:0/2388564661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac07d9b0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.924+0000 7fc5a9e00700 1 --2- 172.21.15.97:0/2388564661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098a0 tx=0x7fc59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74739050cc42ea1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.925+0000 7fc5a8a00700 1 -- 172.21.15.97:0/2388564661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c004160 con 0x7fc5ac07ee00 2024-04-26T21:21:29.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.925+0000 7fc5a8a00700 1 -- 172.21.15.97:0/2388564661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc59c004300 con 0x7fc5ac07ee00 2024-04-26T21:21:29.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.925+0000 7fc5a8a00700 1 -- 172.21.15.97:0/2388564661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c011960 con 0x7fc5ac07ee00 2024-04-26T21:21:29.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 -- 172.21.15.97:0/2388564661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 msgr2=0x7fc5ac07d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/2388564661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098a0 tx=0x7fc59c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 -- 172.21.15.97:0/2388564661 shutdown_connections 2024-04-26T21:21:29.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/2388564661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac07d260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 -- 172.21.15.97:0/2388564661 >> 172.21.15.97:0/2388564661 conn(0x7fc5ac1043f0 msgr2=0x7fc5ac106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 -- 172.21.15.97:0/2388564661 shutdown_connections 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.926+0000 7fc5b0a00700 1 -- 172.21.15.97:0/2388564661 wait complete. 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.927+0000 7fc5b0a00700 1 Processor -- start 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5b0a00700 1 -- start start 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5b0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5b0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10f410 con 0x7fc5ac07ee00 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5a9e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5a9e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57550/0 (socket says 172.21.15.97:57550) 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.928+0000 7fc5a9e00700 1 -- 172.21.15.97:0/1508654711 learned_addr learned my addr 172.21.15.97:0/1508654711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5a9e00700 1 -- 172.21.15.97:0/1508654711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19f450 con 0x7fc5ac07ee00 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5a9e00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc59c011670 tx=0x7fc59c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c01f070 con 0x7fc5ac07ee00 2024-04-26T21:21:29.899 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc59c003b20 con 0x7fc5ac07ee00 2024-04-26T21:21:29.900 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19f6e0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.929+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c0195f0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.930+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a23d0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.931+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc59c022070 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.931+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac064a80 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.931+0000 7fc5a2a00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 0x7fc59003bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.931+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc59c04dd30 con 0x7fc5ac07ee00 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.931+0000 7fc5a9400700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 0x7fc59003bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.932+0000 7fc5a9400700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 0x7fc59003bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc598006fd0 tx=0x7fc598006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:29.902 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:28.937+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc59c01b440 con 0x7fc5ac07ee00 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.163+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc5ac1a25a0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.164+0000 7fc5a2a00700 1 -- 172.21.15.97:0/1508654711 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc59c01baf0 con 0x7fc5ac07ee00 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.166+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 msgr2=0x7fc59003bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.166+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 0x7fc59003bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc598006fd0 tx=0x7fc598006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.166+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 msgr2=0x7fc5ac19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.166+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc59c011670 tx=0x7fc59c011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 shutdown_connections 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc590039750 0x7fc59003bc10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 --2- 172.21.15.97:0/1508654711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5ac07ee00 0x7fc5ac19ef10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 >> 172.21.15.97:0/1508654711 conn(0x7fc5ac1043f0 msgr2=0x7fc5ac104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 shutdown_connections 2024-04-26T21:21:29.903 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:29.167+0000 7fc5b0a00700 1 -- 172.21.15.97:0/1508654711 wait complete. 2024-04-26T21:21:31.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.527+0000 7f7afda00700 1 Processor -- start 2024-04-26T21:21:31.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.527+0000 7f7afda00700 1 -- start start 2024-04-26T21:21:31.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.528+0000 7f7afda00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.528+0000 7f7afda00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7af807cdf0 con 0x7f7af8109910 2024-04-26T21:21:31.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.528+0000 7f7af6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.528+0000 7f7af6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57552/0 (socket says 172.21.15.97:57552) 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.528+0000 7f7af6a00700 1 -- 172.21.15.97:0/986080817 learned_addr learned my addr 172.21.15.97:0/986080817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.529+0000 7f7af6a00700 1 -- 172.21.15.97:0/986080817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af810c240 con 0x7f7af8109910 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.529+0000 7f7af6a00700 1 --2- 172.21.15.97:0/986080817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f18399982af36d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.529+0000 7f7af5600700 1 -- 172.21.15.97:0/986080817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8004160 con 0x7f7af8109910 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.529+0000 7f7af5600700 1 -- 172.21.15.97:0/986080817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae8004300 con 0x7f7af8109910 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.529+0000 7f7af5600700 1 -- 172.21.15.97:0/986080817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8011960 con 0x7f7af8109910 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 -- 172.21.15.97:0/986080817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 msgr2=0x7f7af810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 --2- 172.21.15.97:0/986080817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 -- 172.21.15.97:0/986080817 shutdown_connections 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 --2- 172.21.15.97:0/986080817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af810bd00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 -- 172.21.15.97:0/986080817 >> 172.21.15.97:0/986080817 conn(0x7f7af8103320 msgr2=0x7f7af8105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 -- 172.21.15.97:0/986080817 shutdown_connections 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.530+0000 7f7afda00700 1 -- 172.21.15.97:0/986080817 wait complete. 2024-04-26T21:21:31.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.531+0000 7f7afda00700 1 Processor -- start 2024-04-26T21:21:31.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.531+0000 7f7afda00700 1 -- start start 2024-04-26T21:21:31.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7afda00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7afda00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d1e0 con 0x7f7af8109910 2024-04-26T21:21:31.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7af6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7af6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57556/0 (socket says 172.21.15.97:57556) 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7af6a00700 1 -- 172.21.15.97:0/3520109453 learned_addr learned my addr 172.21.15.97:0/3520109453 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.532+0000 7f7af6a00700 1 -- 172.21.15.97:0/3520109453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819d240 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7af6a00700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8004660 tx=0x7f7ae8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8003aa0 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae801b8d0 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ae8023820 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819d4d0 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.533+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af81a01b0 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.534+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af8064a80 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.534+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7ae801ba80 con 0x7f7af8109910 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.535+0000 7f7aef400700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 0x7f7adc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:31.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.535+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ae804e120 con 0x7f7af8109910 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.535+0000 7f7af6000700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 0x7f7adc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.536+0000 7f7af6000700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 0x7f7adc03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006fd0 tx=0x7f7ae4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.540+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7ae801f030 con 0x7f7af8109910 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.886+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7af81a0760 con 0x7f7af8109910 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.889+0000 7f7aef400700 1 -- 172.21.15.97:0/3520109453 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7ae802bb90 con 0x7f7af8109910 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 msgr2=0x7f7adc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 0x7f7adc03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006fd0 tx=0x7f7ae4006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 msgr2=0x7f7af819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8004660 tx=0x7f7ae8011a30 comp rx=0 tx=0).stop 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 shutdown_connections 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7adc039650 0x7f7adc03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 --2- 172.21.15.97:0/3520109453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af8109910 0x7f7af819cd00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 >> 172.21.15.97:0/3520109453 conn(0x7f7af8103320 msgr2=0x7f7af8103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:31.571 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 shutdown_connections 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-26T21:21:30.893+0000 7f7afda00700 1 -- 172.21.15.97:0/3520109453 wait complete. 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.572 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2024-04-26T21:21:31.573 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.573 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:21:31.573 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:31.573 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2024-04-26T21:21:31.676 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:21:31.676 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:21:31.677 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:21:31.723 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:21:31.724 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:21:31.724 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:21:31.806 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:21:31.806 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:21:31.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:21:31.917 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:21:31.917 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:21:31.917 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:21:32.007 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:21:32.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:21:32.108 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:21:32.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:31 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3520109453' entity='client.admin' 2024-04-26T21:21:32.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:21:32.288 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:21:32.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:21:32.482 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFJh+SFT3VKzX+SuaSOxW2+/4mgniVhwSw5E1RPH1pKi6r1fabLkLbOLXzQAEinLRrXcqFZoWExYf3GSf3PE01DOA5FJhcF0jpl1xDyCp8ba+P+JX6pHC9c1hRrK4Q3KCB2djRsA5Zev6yBkHcCvgwm3bjAQeFCSWUDaUbGDKQp5uXnmdVo4LNPmKDcbhslhElPpithU7z2cWeBbwMrQmk5qGoZInGbkHaZ0pgn1VfZeGDQZqEs1UHY3kze2xev3qYLk2sM49YalMYfjdQEJO29ahx18/vm4q982oyQIxqtd8JqJmEZLABhHANGrgWYI7jsdFUPw347GAsUL2ZO82hvfCjBU7Xkjs0r4ErHmeq884gfK8YvH6354xVvc5SZdchGXgOgNBSsT4pdGPmPCKh65m/agI02ToFHrdjD33Q4tN8UWyx3dkWC7URZS2q1hMnhST6QJnGPR+Srg8jKw7FPQ5iUMNyeOwM4ZOeu0MIw9+sAt2SA698tHa+Vbhw9kM= ceph-9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:21:32.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:21:32.924 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 -- 172.21.15.97:0/3359197698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 msgr2=0x7f69c81054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 --2- 172.21.15.97:0/3359197698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c81054f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009910 tx=0x7f69bc009c20 comp rx=0 tx=0).stop 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 -- 172.21.15.97:0/3359197698 shutdown_connections 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 --2- 172.21.15.97:0/3359197698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c81054f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 -- 172.21.15.97:0/3359197698 >> 172.21.15.97:0/3359197698 conn(0x7f69c8100650 msgr2=0x7f69c8102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 -- 172.21.15.97:0/3359197698 shutdown_connections 2024-04-26T21:21:35.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.488+0000 7f69cfc00700 1 -- 172.21.15.97:0/3359197698 wait complete. 2024-04-26T21:21:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.490+0000 7f69cfc00700 1 Processor -- start 2024-04-26T21:21:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.490+0000 7f69cfc00700 1 -- start start 2024-04-26T21:21:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.490+0000 7f69cfc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.490+0000 7f69cfc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810aec0 con 0x7f69c81050d0 2024-04-26T21:21:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.491+0000 7f69cd200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.491+0000 7f69cd200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47592/0 (socket says 172.21.15.97:47592) 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.491+0000 7f69cd200700 1 -- 172.21.15.97:0/2766334540 learned_addr learned my addr 172.21.15.97:0/2766334540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.491+0000 7f69cd200700 1 -- 172.21.15.97:0/2766334540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c819ab60 con 0x7f69c81050d0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.491+0000 7f69cd200700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69bc005790 tx=0x7f69bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.492+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69bc027070 con 0x7f69c81050d0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.492+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69bc00bdf0 con 0x7f69c81050d0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.492+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c819adf0 con 0x7f69c81050d0 2024-04-26T21:21:35.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.492+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69bc00fe50 con 0x7f69c81050d0 2024-04-26T21:21:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.492+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c819dae0 con 0x7f69c81050d0 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.493+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f69bc02a070 con 0x7f69c81050d0 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.493+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c810b640 con 0x7f69c81050d0 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.493+0000 7f69c5600700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 0x7f69b003bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.493+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f69bc04d940 con 0x7f69c81050d0 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.494+0000 7f69c7e00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 0x7f69b003bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.494+0000 7f69c7e00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 0x7f69b003bd10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:35.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.499+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f69bc013070 con 0x7f69c81050d0 2024-04-26T21:21:35.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.732+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f69c802e5a0 con 0x7f69c81050d0 2024-04-26T21:21:35.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.735+0000 7f69c5600700 1 -- 172.21.15.97:0/2766334540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f69bc0292c0 con 0x7f69c81050d0 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.738+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 msgr2=0x7f69b003bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.738+0000 7f69cfc00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 0x7f69b003bd10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.738+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 msgr2=0x7f69c819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.738+0000 7f69cfc00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69bc005790 tx=0x7f69bc0057c0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 shutdown_connections 2024-04-26T21:21:35.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69b0039850 0x7f69b003bd10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 --2- 172.21.15.97:0/2766334540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69c81050d0 0x7f69c819a620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:35.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 >> 172.21.15.97:0/2766334540 conn(0x7f69c8100650 msgr2=0x7f69c81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:35.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 shutdown_connections 2024-04-26T21:21:35.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:35.739+0000 7f69cfc00700 1 -- 172.21.15.97:0/2766334540 wait complete. 2024-04-26T21:21:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:21:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:36.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:36.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:36.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2766334540' entity='client.admin' 2024-04-26T21:21:36.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:36.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:36.657 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:21:36.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:21:37.064 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 -- 172.21.15.97:0/482254862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 msgr2=0x7f99cc0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 --2- 172.21.15.97:0/482254862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc0758b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009910 tx=0x7f99bc009c20 comp rx=0 tx=0).stop 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 -- 172.21.15.97:0/482254862 shutdown_connections 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 --2- 172.21.15.97:0/482254862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc0758b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 -- 172.21.15.97:0/482254862 >> 172.21.15.97:0/482254862 conn(0x7f99cc0fe420 msgr2=0x7f99cc100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.632+0000 7f99d3000700 1 -- 172.21.15.97:0/482254862 shutdown_connections 2024-04-26T21:21:39.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.633+0000 7f99d3000700 1 -- 172.21.15.97:0/482254862 wait complete. 2024-04-26T21:21:39.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99d3000700 1 Processor -- start 2024-04-26T21:21:39.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99d3000700 1 -- start start 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99d3000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99d3000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc10ae70 con 0x7f99cc077450 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99cbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99cbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47608/0 (socket says 172.21.15.97:47608) 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.634+0000 7f99cbe00700 1 -- 172.21.15.97:0/198414676 learned_addr learned my addr 172.21.15.97:0/198414676 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:39.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99cbe00700 1 -- 172.21.15.97:0/198414676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc19ab40 con 0x7f99cc077450 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99cbe00700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005790 tx=0x7f99bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc027070 con 0x7f99cc077450 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99bc00bdf0 con 0x7f99cc077450 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc19add0 con 0x7f99cc077450 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.635+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99bc00fe50 con 0x7f99cc077450 2024-04-26T21:21:39.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.636+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc19dac0 con 0x7f99cc077450 2024-04-26T21:21:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.637+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f99bc02a070 con 0x7f99cc077450 2024-04-26T21:21:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.637+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc10b5f0 con 0x7f99cc077450 2024-04-26T21:21:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.637+0000 7f99c8c00700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 0x7f99b003bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.637+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f99bc04da00 con 0x7f99cc077450 2024-04-26T21:21:39.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.637+0000 7f99cb400700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 0x7f99b003bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:39.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.638+0000 7f99cb400700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 0x7f99b003bd10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006fd0 tx=0x7f99b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:39.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.643+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99bc013070 con 0x7f99cc077450 2024-04-26T21:21:39.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.875+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f99cc02e5a0 con 0x7f99b0039850 2024-04-26T21:21:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.881+0000 7f99c8c00700 1 -- 172.21.15.97:0/198414676 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99cc02e5a0 con 0x7f99b0039850 2024-04-26T21:21:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 msgr2=0x7f99b003bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 0x7f99b003bd10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006fd0 tx=0x7f99b8006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 msgr2=0x7f99cc19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005790 tx=0x7f99bc0057c0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 shutdown_connections 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f99b0039850 0x7f99b003bd10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 --2- 172.21.15.97:0/198414676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc077450 0x7f99cc19a600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.884+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 >> 172.21.15.97:0/198414676 conn(0x7f99cc0fe420 msgr2=0x7f99cc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.885+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 shutdown_connections 2024-04-26T21:21:39.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:39.885+0000 7f99d3000700 1 -- 172.21.15.97:0/198414676 wait complete. 2024-04-26T21:21:40.600 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-04-26T21:21:40.601 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:21:40.601 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:21:40.658 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:21:40.659 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:21:40.755 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-04-26T21:21:40.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch host add smithi111 2024-04-26T21:21:41.166 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:41.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:41.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:40 smithi097 ceph-mon[28127]: Updating smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:21:42.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:41 smithi097 ceph-mon[28127]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 -- 172.21.15.97:0/3460677689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 msgr2=0x7f78341032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 --2- 172.21.15.97:0/3460677689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341032d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7828009910 tx=0x7f7828009c20 comp rx=0 tx=0).stop 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 -- 172.21.15.97:0/3460677689 shutdown_connections 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 --2- 172.21.15.97:0/3460677689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341032d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 -- 172.21.15.97:0/3460677689 >> 172.21.15.97:0/3460677689 conn(0x7f78340fe7c0 msgr2=0x7f7834100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 -- 172.21.15.97:0/3460677689 shutdown_connections 2024-04-26T21:21:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.101+0000 7f783ac00700 1 -- 172.21.15.97:0/3460677689 wait complete. 2024-04-26T21:21:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.102+0000 7f783ac00700 1 Processor -- start 2024-04-26T21:21:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f783ac00700 1 -- start start 2024-04-26T21:21:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f783ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f783ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f783410d020 con 0x7f7834102ef0 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f7839800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f7839800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58738/0 (socket says 172.21.15.97:58738) 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.103+0000 7f7839800700 1 -- 172.21.15.97:0/385403545 learned_addr learned my addr 172.21.15.97:0/385403545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f7839800700 1 -- 172.21.15.97:0/385403545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78341a0940 con 0x7f7834102ef0 2024-04-26T21:21:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f7839800700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7828006010 tx=0x7f7828005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7828026070 con 0x7f7834102ef0 2024-04-26T21:21:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f782800b6f0 con 0x7f7834102ef0 2024-04-26T21:21:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78341a0c30 con 0x7f7834102ef0 2024-04-26T21:21:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7828003810 con 0x7f7834102ef0 2024-04-26T21:21:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.104+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783419a5c0 con 0x7f7834102ef0 2024-04-26T21:21:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.105+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7828029070 con 0x7f7834102ef0 2024-04-26T21:21:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.105+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78341043d0 con 0x7f7834102ef0 2024-04-26T21:21:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.106+0000 7f7832000700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 0x7f781c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.106+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f782801e070 con 0x7f7834102ef0 2024-04-26T21:21:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.106+0000 7f7838e00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 0x7f781c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.107+0000 7f7838e00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 0x7f781c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:43.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.112+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7828013070 con 0x7f7834102ef0 2024-04-26T21:21:43.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:43.344+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) -- 0x7f783419baf0 con 0x7f781c0396b0 2024-04-26T21:21:43.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:43 smithi097 ceph-mon[28127]: Updating smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:21:43.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:43.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:43.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:43.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:43 smithi097 ceph-mon[28127]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:44.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:44 smithi097 ceph-mon[28127]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:45.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:45 smithi097 ceph-mon[28127]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:45.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:45 smithi097 ceph-mon[28127]: Deploying cephadm binary to smithi111 2024-04-26T21:21:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:47 smithi097 ceph-mon[28127]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:49.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.423+0000 7f7832000700 1 -- 172.21.15.97:0/385403545 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f783419baf0 con 0x7f781c0396b0 2024-04-26T21:21:49.425 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-04-26T21:21:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 msgr2=0x7f781c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 0x7f781c03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 msgr2=0x7f78341a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7828006010 tx=0x7f7828005710 comp rx=0 tx=0).stop 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 shutdown_connections 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f781c0396b0 0x7f781c03bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 --2- 172.21.15.97:0/385403545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834102ef0 0x7f78341a0400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 >> 172.21.15.97:0/385403545 conn(0x7f78340fe7c0 msgr2=0x7f7834075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 shutdown_connections 2024-04-26T21:21:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:49.427+0000 7f783ac00700 1 -- 172.21.15.97:0/385403545 wait complete. 2024-04-26T21:21:49.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:49 smithi097 ceph-mon[28127]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:49.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:49.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:50.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:21:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:50.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:50 smithi097 ceph-mon[28127]: Added host smithi111 2024-04-26T21:21:50.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:51.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:51 smithi097 ceph-mon[28127]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:51.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.686+0000 7f91bac00700 1 -- 172.21.15.97:0/2132883387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 msgr2=0x7f91b4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.686+0000 7f91bac00700 1 --2- 172.21.15.97:0/2132883387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b4104210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009910 tx=0x7f91a8009c20 comp rx=0 tx=0).stop 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.687+0000 7f91bac00700 1 -- 172.21.15.97:0/2132883387 shutdown_connections 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.687+0000 7f91bac00700 1 --2- 172.21.15.97:0/2132883387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b4104210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.687+0000 7f91bac00700 1 -- 172.21.15.97:0/2132883387 >> 172.21.15.97:0/2132883387 conn(0x7f91b40fd6c0 msgr2=0x7f91b40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.687+0000 7f91bac00700 1 -- 172.21.15.97:0/2132883387 shutdown_connections 2024-04-26T21:21:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.687+0000 7f91bac00700 1 -- 172.21.15.97:0/2132883387 wait complete. 2024-04-26T21:21:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.688+0000 7f91bac00700 1 Processor -- start 2024-04-26T21:21:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.688+0000 7f91bac00700 1 -- start start 2024-04-26T21:21:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.688+0000 7f91bac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.688+0000 7f91bac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410d080 con 0x7f91b4103e30 2024-04-26T21:21:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.689+0000 7f91b9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.689+0000 7f91b9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36164/0 (socket says 172.21.15.97:36164) 2024-04-26T21:21:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.689+0000 7f91b9800700 1 -- 172.21.15.97:0/2754181118 learned_addr learned my addr 172.21.15.97:0/2754181118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.689+0000 7f91b9800700 1 -- 172.21.15.97:0/2754181118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b41a0b10 con 0x7f91b4103e30 2024-04-26T21:21:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.689+0000 7f91b9800700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006010 tx=0x7f91a8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.690+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a8026070 con 0x7f91b4103e30 2024-04-26T21:21:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.690+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91a800b750 con 0x7f91b4103e30 2024-04-26T21:21:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.690+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41a0da0 con 0x7f91b4103e30 2024-04-26T21:21:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.690+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a80038a0 con 0x7f91b4103e30 2024-04-26T21:21:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.690+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b419a6a0 con 0x7f91b4103e30 2024-04-26T21:21:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.691+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f91a8029070 con 0x7f91b4103e30 2024-04-26T21:21:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.691+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b406b9b0 con 0x7f91b4103e30 2024-04-26T21:21:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.692+0000 7f91b2000700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 0x7f919c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.692+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f91a801e070 con 0x7f91b4103e30 2024-04-26T21:21:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.692+0000 7f91b8e00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 0x7f919c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.693+0000 7f91b8e00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 0x7f919c03bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:52.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.697+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f91a8013070 con 0x7f91b4103e30 2024-04-26T21:21:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.915+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91b4064a80 con 0x7f919c039700 2024-04-26T21:21:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.917+0000 7f91b2000700 1 -- 172.21.15.97:0/2754181118 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f91b4064a80 con 0x7f919c039700 2024-04-26T21:21:52.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:21:52.919 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 msgr2=0x7f919c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 0x7f919c03bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).stop 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 msgr2=0x7f91b41a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006010 tx=0x7f91a8005710 comp rx=0 tx=0).stop 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 shutdown_connections 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f919c039700 0x7f919c03bbc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 --2- 172.21.15.97:0/2754181118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b4103e30 0x7f91b41a05d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 >> 172.21.15.97:0/2754181118 conn(0x7f91b40fd6c0 msgr2=0x7f91b4108ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 shutdown_connections 2024-04-26T21:21:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:52.921+0000 7f91bac00700 1 -- 172.21.15.97:0/2754181118 wait complete. 2024-04-26T21:21:53.717 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi143 2024-04-26T21:21:53.718 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:21:53.718 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:21:53.775 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:21:53.776 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:21:53.871 INFO:tasks.cephadm:Adding host smithi143 to orchestrator... 2024-04-26T21:21:53.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch host add smithi143 2024-04-26T21:21:53.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:53 smithi097 ceph-mon[28127]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:53.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:54.735 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:54 smithi097 ceph-mon[28127]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:21:55.766 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:55 smithi097 ceph-mon[28127]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.241+0000 7f38b9600700 1 -- 172.21.15.97:0/28535250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 msgr2=0x7f38b410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.241+0000 7f38b9600700 1 --2- 172.21.15.97:0/28535250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b410e7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009910 tx=0x7f38a4009c20 comp rx=0 tx=0).stop 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.241+0000 7f38b9600700 1 -- 172.21.15.97:0/28535250 shutdown_connections 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.242+0000 7f38b9600700 1 --2- 172.21.15.97:0/28535250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b410e7b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.242+0000 7f38b9600700 1 -- 172.21.15.97:0/28535250 >> 172.21.15.97:0/28535250 conn(0x7f38b4078bc0 msgr2=0x7f38b407afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.242+0000 7f38b9600700 1 -- 172.21.15.97:0/28535250 shutdown_connections 2024-04-26T21:21:56.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.242+0000 7f38b9600700 1 -- 172.21.15.97:0/28535250 wait complete. 2024-04-26T21:21:56.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.243+0000 7f38b9600700 1 Processor -- start 2024-04-26T21:21:56.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.243+0000 7f38b9600700 1 -- start start 2024-04-26T21:21:56.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.243+0000 7f38b9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.243+0000 7f38b9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38b41137f0 con 0x7f38b410e3d0 2024-04-26T21:21:56.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.244+0000 7f38b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.244+0000 7f38b2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36180/0 (socket says 172.21.15.97:36180) 2024-04-26T21:21:56.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.244+0000 7f38b2a00700 1 -- 172.21.15.97:0/556102023 learned_addr learned my addr 172.21.15.97:0/556102023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.244+0000 7f38b2a00700 1 -- 172.21.15.97:0/556102023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b419f2f0 con 0x7f38b410e3d0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.244+0000 7f38b2a00700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38a4006010 tx=0x7f38a4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.245+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a4026070 con 0x7f38b410e3d0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.245+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38a400b6f0 con 0x7f38b410e3d0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.245+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b419f5e0 con 0x7f38b410e3d0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.245+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a40038a0 con 0x7f38b410e3d0 2024-04-26T21:21:56.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.245+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b41a3680 con 0x7f38b410e3d0 2024-04-26T21:21:56.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.246+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f38a4029070 con 0x7f38b410e3d0 2024-04-26T21:21:56.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.246+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b410ab60 con 0x7f38b410e3d0 2024-04-26T21:21:56.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.247+0000 7f38ab400700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 0x7f389803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:56.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.247+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f38a401e070 con 0x7f38b410e3d0 2024-04-26T21:21:56.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.247+0000 7f38b2000700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 0x7f389803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:56.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.248+0000 7f38b2000700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 0x7f389803bbc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:56.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.253+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f38a4013070 con 0x7f38b410e3d0 2024-04-26T21:21:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:21:56.484+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}) -- 0x7f38b41a0110 con 0x7f3898039700 2024-04-26T21:21:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:57 smithi097 ceph-mon[28127]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:57 smithi097 ceph-mon[28127]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: Deploying cephadm binary to smithi143 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:00.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:59 smithi097 ceph-mon[28127]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:21:59 smithi097 ceph-mon[28127]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-26T21:22:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:01 smithi097 ceph-mon[28127]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:02.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.465+0000 7f38ab400700 1 -- 172.21.15.97:0/556102023 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f38b41a0110 con 0x7f3898039700 2024-04-26T21:22:02.468 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi143' with addr '172.21.15.143' 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.469+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 msgr2=0x7f389803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.469+0000 7f38b9600700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 0x7f389803bbc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.469+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 msgr2=0x7f38b419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.469+0000 7f38b9600700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38a4006010 tx=0x7f38a4005710 comp rx=0 tx=0).stop 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 shutdown_connections 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3898039700 0x7f389803bbc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 --2- 172.21.15.97:0/556102023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38b410e3d0 0x7f38b419edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 >> 172.21.15.97:0/556102023 conn(0x7f38b4078bc0 msgr2=0x7f38b4110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 shutdown_connections 2024-04-26T21:22:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:02.470+0000 7f38b9600700 1 -- 172.21.15.97:0/556102023 wait complete. 2024-04-26T21:22:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:02 smithi097 ceph-mon[28127]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:03.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:22:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:03.939 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:03 smithi097 ceph-mon[28127]: Added host smithi143 2024-04-26T21:22:03.939 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:03 smithi097 ceph-mon[28127]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:03.939 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:03 smithi097 ceph-mon[28127]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:04.971 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:04 smithi097 ceph-mon[28127]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:22:04.971 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:04.971 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:04.971 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:04.971 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 -- 172.21.15.97:0/1275459483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 msgr2=0x7f8540103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 --2- 172.21.15.97:0/1275459483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f8540103470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8530009910 tx=0x7f8530009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 -- 172.21.15.97:0/1275459483 shutdown_connections 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 --2- 172.21.15.97:0/1275459483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f8540103470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 -- 172.21.15.97:0/1275459483 >> 172.21.15.97:0/1275459483 conn(0x7f85400fe960 msgr2=0x7f8540100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 -- 172.21.15.97:0/1275459483 shutdown_connections 2024-04-26T21:22:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.814+0000 7f8546a00700 1 -- 172.21.15.97:0/1275459483 wait complete. 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.815+0000 7f8546a00700 1 Processor -- start 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f8546a00700 1 -- start start 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f8546a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f8546a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f854010d1c0 con 0x7f8540103090 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f853fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f853fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53414/0 (socket says 172.21.15.97:53414) 2024-04-26T21:22:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.816+0000 7f853fe00700 1 -- 172.21.15.97:0/2239551184 learned_addr learned my addr 172.21.15.97:0/2239551184 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:22:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f853fe00700 1 -- 172.21.15.97:0/2239551184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85401a0c50 con 0x7f8540103090 2024-04-26T21:22:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f853fe00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f8530005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8530026070 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f853000b6f0 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401a0f40 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8530003810 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.817+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854019a8d0 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.818+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8540104570 con 0x7f8540103090 2024-04-26T21:22:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.819+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8530029070 con 0x7f8540103090 2024-04-26T21:22:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.819+0000 7f853cc00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 0x7f852403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.819+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f853001e070 con 0x7f8540103090 2024-04-26T21:22:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.819+0000 7f853f400700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 0x7f852403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.820+0000 7f853f400700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 0x7f852403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f852c006fd0 tx=0x7f852c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:05.825+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8530013070 con 0x7f8540103090 2024-04-26T21:22:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.058+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f854019be00 con 0x7f85240396b0 2024-04-26T21:22:06.060 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:05 smithi097 ceph-mon[28127]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:06.060 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.060+0000 7f853cc00700 1 -- 172.21.15.97:0/2239551184 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f854019be00 con 0x7f85240396b0 2024-04-26T21:22:06.061 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:22:06.062 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}] 2024-04-26T21:22:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.063+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 msgr2=0x7f852403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.063+0000 7f8546a00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 0x7f852403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f852c006fd0 tx=0x7f852c006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.063+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 msgr2=0x7f85401a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.063+0000 7f8546a00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f8530005710 comp rx=0 tx=0).stop 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.063+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 shutdown_connections 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.064+0000 7f8546a00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f85240396b0 0x7f852403bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.064+0000 7f8546a00700 1 --2- 172.21.15.97:0/2239551184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8540103090 0x7f85401a0710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.064+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 >> 172.21.15.97:0/2239551184 conn(0x7f85400fe960 msgr2=0x7f854010ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.064+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 shutdown_connections 2024-04-26T21:22:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:06.064+0000 7f8546a00700 1 -- 172.21.15.97:0/2239551184 wait complete. 2024-04-26T21:22:06.872 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:22:06.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:22:07.304 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:07 smithi097 ceph-mon[28127]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:22:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:07 smithi097 ceph-mon[28127]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 -- 172.21.15.97:0/4094164781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 msgr2=0x7f2004105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 --2- 172.21.15.97:0/4094164781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f2004105690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009910 tx=0x7f1ff8009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:09.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 -- 172.21.15.97:0/4094164781 shutdown_connections 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 --2- 172.21.15.97:0/4094164781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f2004105690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 -- 172.21.15.97:0/4094164781 >> 172.21.15.97:0/4094164781 conn(0x7f2004100b40 msgr2=0x7f2004102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 -- 172.21.15.97:0/4094164781 shutdown_connections 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.336+0000 7f200c000700 1 -- 172.21.15.97:0/4094164781 wait complete. 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.337+0000 7f200c000700 1 Processor -- start 2024-04-26T21:22:09.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.337+0000 7f200c000700 1 -- start start 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f200c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f200c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f200410f3e0 con 0x7f20041052b0 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f2009600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f2009600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53426/0 (socket says 172.21.15.97:53426) 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f2009600700 1 -- 172.21.15.97:0/2197153211 learned_addr learned my addr 172.21.15.97:0/2197153211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.338+0000 7f2009600700 1 -- 172.21.15.97:0/2197153211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041a0b60 con 0x7f20041052b0 2024-04-26T21:22:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f2009600700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006010 tx=0x7f1ff8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ff8026070 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ff800b6f0 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ff8003810 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a0e50 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.339+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200419a7a0 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.340+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2004106790 con 0x7f20041052b0 2024-04-26T21:22:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.340+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ff8029070 con 0x7f20041052b0 2024-04-26T21:22:09.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.341+0000 7f2002000700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 0x7f1fec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:09.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.341+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ff801e070 con 0x7f20041052b0 2024-04-26T21:22:09.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.341+0000 7f2008c00700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 0x7f1fec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.342+0000 7f2008c00700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 0x7f1fec03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.346+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ff8013070 con 0x7f20041052b0 2024-04-26T21:22:09.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:09 smithi097 ceph-mon[28127]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:09.576+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f200419bcd0 con 0x7f20041052b0 2024-04-26T21:22:10.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.284+0000 7f2002000700 1 -- 172.21.15.97:0/2197153211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1ff802ade0 con 0x7f20041052b0 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.287+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 msgr2=0x7f1fec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.287+0000 7f200c000700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 0x7f1fec03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.287+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 msgr2=0x7f20041a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.287+0000 7f200c000700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006010 tx=0x7f1ff8005710 comp rx=0 tx=0).stop 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 shutdown_connections 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fec0396b0 0x7f1fec03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 --2- 172.21.15.97:0/2197153211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20041052b0 0x7f20041a0620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 >> 172.21.15.97:0/2197153211 conn(0x7f2004100b40 msgr2=0x7f2004106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:10.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 shutdown_connections 2024-04-26T21:22:10.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:10.288+0000 7f200c000700 1 -- 172.21.15.97:0/2197153211 wait complete. 2024-04-26T21:22:10.291 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-04-26T21:22:10.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:10 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2197153211' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:22:11.068 INFO:tasks.cephadm:Adding mon.a on smithi097 2024-04-26T21:22:11.068 INFO:tasks.cephadm:Adding mon.b on smithi111 2024-04-26T21:22:11.068 INFO:tasks.cephadm:Adding mon.c on smithi143 2024-04-26T21:22:11.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c' 2024-04-26T21:22:11.469 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:22:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:11 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2197153211' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:22:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:11 smithi097 ceph-mon[28127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:11 smithi097 ceph-mon[28127]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:11.575 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:22:13.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:13.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:13.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:13.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:13.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:13.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:13.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:13.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 -- 172.21.15.143:0/3391245194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 msgr2=0x7f7c98105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3391245194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c98105d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009910 tx=0x7f7c88009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 -- 172.21.15.143:0/3391245194 shutdown_connections 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3391245194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c98105d20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 -- 172.21.15.143:0/3391245194 >> 172.21.15.143:0/3391245194 conn(0x7f7c980fd360 msgr2=0x7f7c980ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 -- 172.21.15.143:0/3391245194 shutdown_connections 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.305+0000 7f7c9d600700 1 -- 172.21.15.143:0/3391245194 wait complete. 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.306+0000 7f7c9d600700 1 Processor -- start 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.306+0000 7f7c9d600700 1 -- start start 2024-04-26T21:22:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.307+0000 7f7c9d600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.307+0000 7f7c9d600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98108c30 con 0x7f7c98103930 2024-04-26T21:22:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.307+0000 7f7c96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.307+0000 7f7c96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.143:51628/0 (socket says 172.21.15.143:51628) 2024-04-26T21:22:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.307+0000 7f7c96a00700 1 -- 172.21.15.143:0/3075855166 learned_addr learned my addr 172.21.15.143:0/3075855166 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.308+0000 7f7c96a00700 1 -- 172.21.15.143:0/3075855166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9819ab90 con 0x7f7c98103930 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.308+0000 7f7c96a00700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c88005b40 tx=0x7f7c88005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.309+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c88027070 con 0x7f7c98103930 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.309+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9819ae80 con 0x7f7c98103930 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.309+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9819db10 con 0x7f7c98103930 2024-04-26T21:22:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.309+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c8800bdf0 con 0x7f7c98103930 2024-04-26T21:22:14.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.309+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c88022bd0 con 0x7f7c98103930 2024-04-26T21:22:14.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.310+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c78005360 con 0x7f7c98103930 2024-04-26T21:22:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.311+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7c8802a070 con 0x7f7c98103930 2024-04-26T21:22:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.311+0000 7f7c8f400700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 0x7f7c7c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.311+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7c8801e030 con 0x7f7c98103930 2024-04-26T21:22:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.312+0000 7f7c96000700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 0x7f7c7c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:14.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.313+0000 7f7c96000700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 0x7f7c7c03bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:14.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.328+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c88013070 con 0x7f7c98103930 2024-04-26T21:22:14.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.584+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}) -- 0x7f7c78000cb0 con 0x7f7c7c0396b0 2024-04-26T21:22:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.599+0000 7f7c8f400700 1 -- 172.21.15.143:0/3075855166 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c78000cb0 con 0x7f7c7c0396b0 2024-04-26T21:22:14.600 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mon update... 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 msgr2=0x7f7c7c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 0x7f7c7c03bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 msgr2=0x7f7c9819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c88005b40 tx=0x7f7c88005810 comp rx=0 tx=0).stop 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 shutdown_connections 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7c7c0396b0 0x7f7c7c03bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 --2- 172.21.15.143:0/3075855166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c98103930 0x7f7c9819a650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 >> 172.21.15.143:0/3075855166 conn(0x7f7c980fd360 msgr2=0x7f7c980fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 shutdown_connections 2024-04-26T21:22:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:14.602+0000 7f7c9d600700 1 -- 172.21.15.143:0/3075855166 wait complete. 2024-04-26T21:22:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:14 smithi097 ceph-mon[28127]: Updating smithi143:/etc/ceph/ceph.conf 2024-04-26T21:22:15.342 DEBUG:teuthology.orchestra.run.smithi111:mon.b> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:22:15.351 DEBUG:teuthology.orchestra.run.smithi143:mon.c> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:22:15.355 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:22:15.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:22:15.775 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c;count:3 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:15 smithi097 ceph-mon[28127]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:17.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 -- 172.21.15.143:0/4016278193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 msgr2=0x7fbb2c1054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:17.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 --2- 172.21.15.143:0/4016278193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1054d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009910 tx=0x7fbb20009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:17.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 -- 172.21.15.143:0/4016278193 shutdown_connections 2024-04-26T21:22:17.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 --2- 172.21.15.143:0/4016278193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1054d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:17.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 -- 172.21.15.143:0/4016278193 >> 172.21.15.143:0/4016278193 conn(0x7fbb2c1009a0 msgr2=0x7fbb2c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:17.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 -- 172.21.15.143:0/4016278193 shutdown_connections 2024-04-26T21:22:17.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.824+0000 7fbb32400700 1 -- 172.21.15.143:0/4016278193 wait complete. 2024-04-26T21:22:17.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.825+0000 7fbb32400700 1 Processor -- start 2024-04-26T21:22:17.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.826+0000 7fbb32400700 1 -- start start 2024-04-26T21:22:17.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.826+0000 7fbb32400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.826+0000 7fbb32400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c10f220 con 0x7fbb2c1050f0 2024-04-26T21:22:17.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.827+0000 7fbb31000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.827+0000 7fbb31000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.143:48534/0 (socket says 172.21.15.143:48534) 2024-04-26T21:22:17.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.827+0000 7fbb31000700 1 -- 172.21.15.143:0/3656970058 learned_addr learned my addr 172.21.15.143:0/3656970058 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:17.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.827+0000 7fbb31000700 1 -- 172.21.15.143:0/3656970058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c1a0af0 con 0x7fbb2c1050f0 2024-04-26T21:22:17.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.828+0000 7fbb31000700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005660 tx=0x7fbb20005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.828+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb20026070 con 0x7fbb2c1050f0 2024-04-26T21:22:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.828+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb2000b6f0 con 0x7fbb2c1050f0 2024-04-26T21:22:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.829+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c1a0de0 con 0x7fbb2c1050f0 2024-04-26T21:22:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.829+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb200037b0 con 0x7fbb2c1050f0 2024-04-26T21:22:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.829+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c19a850 con 0x7fbb2c1050f0 2024-04-26T21:22:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.830+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c1065d0 con 0x7fbb2c1050f0 2024-04-26T21:22:17.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.830+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbb20029070 con 0x7fbb2c1050f0 2024-04-26T21:22:17.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.831+0000 7fbb29600700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 0x7fbb1403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:17.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.831+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbb2001e070 con 0x7fbb2c1050f0 2024-04-26T21:22:17.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.832+0000 7fbb2be00700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 0x7fbb1403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.833+0000 7fbb2be00700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 0x7fbb1403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006fd0 tx=0x7fbb18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:17.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:17.846+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb20013070 con 0x7fbb2c1050f0 2024-04-26T21:22:18.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:17 smithi097 ceph-mon[28127]: Deploying daemon mon.c on smithi143 2024-04-26T21:22:18.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:17 smithi097 ceph-mon[28127]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:18.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.174+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbb2c0008d0 con 0x7fbb2c1050f0 2024-04-26T21:22:18.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.175+0000 7fbb29600700 1 -- 172.21.15.143:0/3656970058 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fbb20030030 con 0x7fbb2c1050f0 2024-04-26T21:22:18.176 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:22:18.176 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":1,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","modified":"2024-04-26T21:20:09.486150Z","created":"2024-04-26T21:20:09.486150Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 msgr2=0x7fbb1403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 0x7fbb1403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006fd0 tx=0x7fbb18006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 msgr2=0x7fbb2c1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005660 tx=0x7fbb20005740 comp rx=0 tx=0).stop 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 shutdown_connections 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbb140396b0 0x7fbb1403bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 --2- 172.21.15.143:0/3656970058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2c1050f0 0x7fbb2c1a05b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 >> 172.21.15.143:0/3656970058 conn(0x7fbb2c1009a0 msgr2=0x7fbb2c106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 shutdown_connections 2024-04-26T21:22:18.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:18.178+0000 7fbb32400700 1 -- 172.21.15.143:0/3656970058 wait complete. 2024-04-26T21:22:18.180 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 1 2024-04-26T21:22:19.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:18 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/3656970058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:19.965 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:19 smithi097 ceph-mon[28127]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:20.164 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:22:20.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:22:20.828 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:21 smithi097 ceph-mon[28127]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.288 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 podman[31435]: 2024-04-26 21:22:23.15794205 +0000 UTC m=+0.222306834 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:22:23.288 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 podman[31435]: 2024-04-26 21:22:23.286223975 +0000 UTC m=+0.350588803 container create 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:22:23.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 podman[31435]: 2024-04-26 21:22:23.626173618 +0000 UTC m=+0.690538419 container init 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 podman[31435]: 2024-04-26 21:22:23.635435573 +0000 UTC m=+0.699800359 container start 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pidfile_write: ignore empty --pid-file 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: load: jerasure load: lrc 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Git sha 0 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: DB SUMMARY 2024-04-26T21:22:23.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: DB Session ID: YUH3HVJXSDKC7K87CRE9 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:22:23.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:22:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.env: 0x55999b6efcc0 2024-04-26T21:22:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:22:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.info_log: 0x5599d7225dc0 2024-04-26T21:22:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.statistics: (nil) 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.use_fsync: 0 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:22:23.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.db_log_dir: 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.wal_dir: 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:22:23.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.write_buffer_manager: 0x5599d71f70e0 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:22:23.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.unordered_write: 0 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:22:23.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.row_cache: None 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.wal_filter: None 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.wal_compression: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:22:23.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:22:23.805 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:22:23.806 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_open_files: -1 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Compression algorithms supported: 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kZSTD supported: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:22:23.807 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.merge_operator: 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_filter: None 2024-04-26T21:22:23.808 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5599d7225f80) 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_type: 0 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_index_type: 0 2024-04-26T21:22:23.809 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_shortening: 1 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: checksum: 4 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: no_block_cache: 0 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache: 0x5599d72291f0 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_options: 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: capacity : 536870912 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: num_shard_bits : 4 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: strict_capacity_limit : 0 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:22:23.810 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_compressed: (nil) 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: persistent_cache: (nil) 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size: 4096 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size_deviation: 10 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_restart_interval: 16 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_block_restart_interval: 1 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: metadata_block_size: 4096 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: partition_filters: 0 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: use_delta_encoding: 1 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: filter_policy: bloomfilter 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: whole_key_filtering: 1 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: verify_compression: 0 2024-04-26T21:22:23.811 INFO:journalctl@ceph.mon.c.smithi143.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: format_version: 5 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_index_compression: 1 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_align: 0 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: max_auto_readahead_size: 262144 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: prepopulate_block_cache: 0 2024-04-26T21:22:23.812 INFO:journalctl@ceph.mon.c.smithi143.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression: NoCompression 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:22:23.814 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.num_levels: 7 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:22:23.815 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:22:23.816 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:22:23.817 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:22:23.818 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:22:23.819 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:22:23.820 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.ttl: 2592000 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:22:23.821 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:22:23.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fb162680-b66d-4f0d-8e65-b799bf2d0cf6 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166543690742, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166543691733, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714166543, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fb162680-b66d-4f0d-8e65-b799bf2d0cf6", "db_session_id": "YUH3HVJXSDKC7K87CRE9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714166543692000, "job": 1, "event": "recovery_finished"} 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5599d7346000 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: DB pointer 0x5599d7332000 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:22:23.823 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** DB Stats ** 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.824 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:22:23.825 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:22:23.826 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: Block cache BinnedLRUCache@0x5599d72291f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: using public_addr v2:172.21.15.143:0/0 -> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: starting mon.c rank -1 at public addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] at bind addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(???) e0 preinit fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 bash[31435]: 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 systemd[1]: Started Ceph mon.c for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T21:22:23.827 INFO:journalctl@ceph.mon.c.smithi143.stdout: e1 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout: legacy client fscid: -1 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout: No filesystems configured 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:22:23.828 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Added host smithi111 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.829 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Deploying cephadm binary to smithi143 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:23.830 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Added host smithi143 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:22:23.831 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.832 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2197153211' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2197153211' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:22:23.833 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi143:/etc/ceph/ceph.conf 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c;count:3 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:22:23.834 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: Deploying daemon mon.c on smithi143 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/3656970058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.835 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:23.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:23 smithi143 ceph-mon[31461]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 -- 172.21.15.143:0/3094490957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 msgr2=0x7f2d10103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 --2- 172.21.15.143:0/3094490957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d10103540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009910 tx=0x7f2d04009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 -- 172.21.15.143:0/3094490957 shutdown_connections 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 --2- 172.21.15.143:0/3094490957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d10103540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 -- 172.21.15.143:0/3094490957 >> 172.21.15.143:0/3094490957 conn(0x7f2d100fe9f0 msgr2=0x7f2d10100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 -- 172.21.15.143:0/3094490957 shutdown_connections 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.716+0000 7f2d17e00700 1 -- 172.21.15.143:0/3094490957 wait complete. 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.718+0000 7f2d17e00700 1 Processor -- start 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.718+0000 7f2d17e00700 1 -- start start 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.718+0000 7f2d17e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.718+0000 7f2d17e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1010d290 con 0x7f2d10103160 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.719+0000 7f2d15400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:23.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.719+0000 7f2d15400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.143:48554/0 (socket says 172.21.15.143:48554) 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.719+0000 7f2d15400700 1 -- 172.21.15.143:0/2276267768 learned_addr learned my addr 172.21.15.143:0/2276267768 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.720+0000 7f2d15400700 1 -- 172.21.15.143:0/2276267768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d101a0c90 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.720+0000 7f2d15400700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.721+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d04026070 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.721+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d101a0f80 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.721+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1019a910 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.721+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d0400b6f0 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.721+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d04003810 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.722+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf0005330 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.723+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2d04029070 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.723+0000 7f2d0e000700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 0x7f2cf803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.723+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2d0401e070 con 0x7f2d10103160 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.724+0000 7f2d14a00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 0x7f2cf803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:23.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.726+0000 7f2d14a00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 0x7f2cf803bb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:23.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.741+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2d04013070 con 0x7f2d10103160 2024-04-26T21:22:23.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:23.756+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2d04029360 con 0x7f2d10103160 2024-04-26T21:22:24.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:23 smithi097 ceph-mon[28127]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:24.071+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2cf0005620 con 0x7f2d10103160 2024-04-26T21:22:28.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.785+0000 7f2d0e000700 1 -- 172.21.15.143:0/2276267768 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f2d040282c0 con 0x7f2d10103160 2024-04-26T21:22:28.786 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:22:28.787 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","modified":"2024-04-26T21:22:23.752496Z","created":"2024-04-26T21:20:09.486150Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 msgr2=0x7f2cf803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 0x7f2cf803bb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc006e40 comp rx=0 tx=0).stop 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 msgr2=0x7f2d101a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04005710 comp rx=0 tx=0).stop 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 shutdown_connections 2024-04-26T21:22:28.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cf80396b0 0x7f2cf803bb70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:28.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 --2- 172.21.15.143:0/2276267768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d10103160 0x7f2d101a0750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:28.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.789+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 >> 172.21.15.143:0/2276267768 conn(0x7f2d100fe9f0 msgr2=0x7f2d10075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:28.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.790+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 shutdown_connections 2024-04-26T21:22:28.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:28.790+0000 7f2d17e00700 1 -- 172.21.15.143:0/2276267768 wait complete. 2024-04-26T21:22:28.792 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: mon.a calling monitor election 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/2276267768' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: mon.c calling monitor election 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: monmap epoch 2 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: last_changed 2024-04-26T21:22:23.752496+0000 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: min_mon_release 19 (squid) 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: election_strategy: 1 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: fsmap 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:29.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: mgrmap e13: x(active, since 68s) 2024-04-26T21:22:29.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: overall HEALTH_OK 2024-04-26T21:22:29.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:29.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: mon.a calling monitor election 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/2276267768' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: mon.c calling monitor election 2024-04-26T21:22:29.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: monmap epoch 2 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: last_changed 2024-04-26T21:22:23.752496+0000 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: min_mon_release 19 (squid) 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: election_strategy: 1 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: fsmap 2024-04-26T21:22:29.317 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:29.318 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: mgrmap e13: x(active, since 68s) 2024-04-26T21:22:29.318 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: overall HEALTH_OK 2024-04-26T21:22:29.318 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:29.318 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:29 smithi143 ceph-mon[31461]: Deploying daemon mon.b on smithi111 2024-04-26T21:22:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:29 smithi097 ceph-mon[28127]: Deploying daemon mon.b on smithi111 2024-04-26T21:22:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:30.650 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:22:30.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:22:31.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:30 smithi097 ceph-mon[28127]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:31.020 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:22:30 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:22:30.753+0000 7f3adea00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:22:31.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:30 smithi143 ceph-mon[31461]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:31.080 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:33.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 -- 172.21.15.143:0/893271680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e8109850 msgr2=0x7fb1e8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 --2- 172.21.15.143:0/893271680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e8109850 0x7fb1e8109c30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009910 tx=0x7fb1dc009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 -- 172.21.15.143:0/893271680 shutdown_connections 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 --2- 172.21.15.143:0/893271680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e8109850 0x7fb1e8109c30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 -- 172.21.15.143:0/893271680 >> 172.21.15.143:0/893271680 conn(0x7fb1e8078da0 msgr2=0x7fb1e807b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.109+0000 7fb1efc00700 1 -- 172.21.15.143:0/893271680 shutdown_connections 2024-04-26T21:22:33.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.110+0000 7fb1efc00700 1 -- 172.21.15.143:0/893271680 wait complete. 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.111+0000 7fb1efc00700 1 Processor -- start 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.111+0000 7fb1efc00700 1 -- start start 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1efc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1efc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 0x7fb1e81a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1efc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e81138e0 con 0x7fb1e819f4d0 2024-04-26T21:22:33.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1efc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8113760 con 0x7fb1e8109850 2024-04-26T21:22:33.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:33.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1e7e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 0x7fb1e81a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:33.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48192/0 (socket says 172.21.15.143:48192) 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 -- 172.21.15.143:0/1757700975 learned_addr learned my addr 172.21.15.143:0/1757700975 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 -- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 msgr2=0x7fb1e819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 -- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 msgr2=0x7fb1e819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.112+0000 7fb1ed200700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.113+0000 7fb1ed200700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.113+0000 7fb1e7e00700 1 -- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 msgr2=0x7fb1e819ef90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.113+0000 7fb1e7e00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:33.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.113+0000 7fb1e7e00700 1 -- 172.21.15.143:0/1757700975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e819fb10 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.113+0000 7fb1e7e00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 0x7fb1e81a3860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b4d0 tx=0x7fb1d400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.114+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb1d4011630 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.114+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e81a3da0 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.114+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1d4011c60 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.114+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb1d4010c90 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.114+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81a4380 con 0x7fb1e819f4d0 2024-04-26T21:22:33.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.115+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e81a4690 con 0x7fb1e819f4d0 2024-04-26T21:22:33.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.116+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb1d4018020 con 0x7fb1e819f4d0 2024-04-26T21:22:33.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.116+0000 7fb1e5600700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 0x7fb1d003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:33.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.116+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb1d404c390 con 0x7fb1e819f4d0 2024-04-26T21:22:33.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.117+0000 7fb1ed200700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 0x7fb1d003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:33.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.118+0000 7fb1ed200700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 0x7fb1d003b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009750 tx=0x7fb1dc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.132+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb1e81a4690 con 0x7fb1e819f4d0 2024-04-26T21:22:33.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.446+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb1e81a02f0 con 0x7fb1e819f4d0 2024-04-26T21:22:33.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.448+0000 7fb1e5600700 1 -- 172.21.15.143:0/1757700975 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb1d401f020 con 0x7fb1e819f4d0 2024-04-26T21:22:33.475 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:22:33.475 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","modified":"2024-04-26T21:22:23.752496Z","created":"2024-04-26T21:20:09.486150Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.450+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 msgr2=0x7fb1d003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.450+0000 7fb1efc00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 0x7fb1d003b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009750 tx=0x7fb1dc005a90 comp rx=0 tx=0).stop 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 msgr2=0x7fb1e81a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 0x7fb1e81a3860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b4d0 tx=0x7fb1d400b890 comp rx=0 tx=0).stop 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 shutdown_connections 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb1d0039440 0x7fb1d003b900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1e8109850 0x7fb1e819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 --2- 172.21.15.143:0/1757700975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1e819f4d0 0x7fb1e81a3860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 >> 172.21.15.143:0/1757700975 conn(0x7fb1e8078da0 msgr2=0x7fb1e810b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:33.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 shutdown_connections 2024-04-26T21:22:33.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:33.451+0000 7fb1efc00700 1 -- 172.21.15.143:0/1757700975 wait complete. 2024-04-26T21:22:33.477 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-04-26T21:22:33.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:33 smithi143 ceph-mon[31461]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:33.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/1757700975' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:33 smithi097 ceph-mon[28127]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/1757700975' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:22:34.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:34 smithi111 ceph-mon[31001]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-26T21:22:35.184 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:22:35.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:22:35.603 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:22:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: mon.a calling monitor election 2024-04-26T21:22:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: mon.c calling monitor election 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: monmap epoch 3 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: min_mon_release 19 (squid) 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: election_strategy: 1 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: fsmap 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: mgrmap e13: x(active, since 79s) 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: overall HEALTH_OK 2024-04-26T21:22:40.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:22:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: mon.a calling monitor election 2024-04-26T21:22:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: mon.c calling monitor election 2024-04-26T21:22:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:40.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:22:40.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: monmap epoch 3 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: min_mon_release 19 (squid) 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: election_strategy: 1 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:40.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:40.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:40.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: fsmap 2024-04-26T21:22:40.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:40.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: mgrmap e13: x(active, since 79s) 2024-04-26T21:22:40.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: overall HEALTH_OK 2024-04-26T21:22:40.069 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.069 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.069 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 -- 172.21.15.143:0/947885378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c1031c0 msgr2=0x7fbc30005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:40.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 --2- 172.21.15.143:0/947885378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c1031c0 0x7fbc30005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005fa0 tx=0x7fbc3c00ffc0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 -- 172.21.15.143:0/947885378 shutdown_connections 2024-04-26T21:22:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 --2- 172.21.15.143:0/947885378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c1031c0 0x7fbc30005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 -- 172.21.15.143:0/947885378 >> 172.21.15.143:0/947885378 conn(0x7fbc4c0fea10 msgr2=0x7fbc4c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 -- 172.21.15.143:0/947885378 shutdown_connections 2024-04-26T21:22:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.490+0000 7fbc52e00700 1 -- 172.21.15.143:0/947885378 wait complete. 2024-04-26T21:22:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.494+0000 7fbc52e00700 1 Processor -- start 2024-04-26T21:22:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.495+0000 7fbc52e00700 1 -- start start 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.495+0000 7fbc52e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.495+0000 7fbc52e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c075c60 0x7fbc4c0760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc52e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc52e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c10ad00 con 0x7fbc4c075c60 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc52e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c10ab80 con 0x7fbc4c1031c0 2024-04-26T21:22:40.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc52e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c10ae80 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc50e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc50e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55594/0 (socket says 172.21.15.143:55594) 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc50e00700 1 -- 172.21.15.143:0/3537341433 learned_addr learned my addr 172.21.15.143:0/3537341433 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc4b400700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c075c60 0x7fbc4c0760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.496+0000 7fbc4be00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc4be00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 msgr2=0x7fbc4c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc4be00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 msgr2=0x7fbc4c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc4be00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 msgr2=0x7fbc4c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc4be00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c075c60 msgr2=0x7fbc4c0760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c075c60 0x7fbc4c0760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 -- 172.21.15.143:0/3537341433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c1a7710 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc50e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400ba50 tx=0x7fbc4400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:40.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.497+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44019070 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.498+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc4400cdc0 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.498+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c1a7a00 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.498+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44007350 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.498+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c1a7f90 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.499+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbc4401c070 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.499+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4c064a80 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.500+0000 7fbc48c00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 0x7fbc2c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:40.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.500+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbc4404be50 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.501+0000 7fbc4be00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 0x7fbc2c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:40.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.502+0000 7fbc4be00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 0x7fbc2c03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00fb20 tx=0x7fbc3c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:40.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.505+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbc44007d90 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:40.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:40.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:40.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.839+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbc4c02e490 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.839+0000 7fbc48c00700 1 -- 172.21.15.143:0/3537341433 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fbc44012500 con 0x7fbc4c1a4bf0 2024-04-26T21:22:40.840 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-04-26T21:22:40.840 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","modified":"2024-04-26T21:22:34.498573Z","created":"2024-04-26T21:20:09.486150Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 msgr2=0x7fbc2c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 0x7fbc2c03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00fb20 tx=0x7fbc3c005da0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 msgr2=0x7fbc4c1a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400ba50 tx=0x7fbc4400bd60 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 shutdown_connections 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbc2c039ae0 0x7fbc2c03bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4c1031c0 0x7fbc4c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc4c1a4bf0 0x7fbc4c1a6fe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 --2- 172.21.15.143:0/3537341433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c075c60 0x7fbc4c0760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 >> 172.21.15.143:0/3537341433 conn(0x7fbc4c0fea10 msgr2=0x7fbc4c0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 shutdown_connections 2024-04-26T21:22:40.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:40.842+0000 7fbc52e00700 1 -- 172.21.15.143:0/3537341433 wait complete. 2024-04-26T21:22:40.844 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 3 2024-04-26T21:22:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.548 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:22:41.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.a calling monitor election 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.c calling monitor election 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:41.600 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: monmap epoch 3 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: min_mon_release 19 (squid) 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: election_strategy: 1 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: fsmap 2024-04-26T21:22:41.601 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mgrmap e13: x(active, since 79s) 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: overall HEALTH_OK 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.b calling monitor election 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.b calling monitor election 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.c calling monitor election 2024-04-26T21:22:41.602 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.a calling monitor election 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: monmap epoch 3 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:41.603 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: min_mon_release 19 (squid) 2024-04-26T21:22:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mon.b calling monitor election 2024-04-26T21:22:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mon.b calling monitor election 2024-04-26T21:22:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mon.c calling monitor election 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mon.a calling monitor election 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: monmap epoch 3 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: min_mon_release 19 (squid) 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: election_strategy: 1 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: fsmap 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:41.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: mgrmap e13: x(active, since 81s) 2024-04-26T21:22:41.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:41 smithi143 ceph-mon[31461]: overall HEALTH_OK 2024-04-26T21:22:41.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: election_strategy: 1 2024-04-26T21:22:41.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:41.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:41.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:41.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: fsmap 2024-04-26T21:22:41.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:41.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: mgrmap e13: x(active, since 81s) 2024-04-26T21:22:41.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:41 smithi111 ceph-mon[31001]: overall HEALTH_OK 2024-04-26T21:22:41.967 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mon.b calling monitor election 2024-04-26T21:22:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mon.b calling monitor election 2024-04-26T21:22:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mon.c calling monitor election 2024-04-26T21:22:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mon.a calling monitor election 2024-04-26T21:22:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: monmap epoch 3 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: last_changed 2024-04-26T21:22:34.498573+0000 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: created 2024-04-26T21:20:09.486150+0000 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: min_mon_release 19 (squid) 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: election_strategy: 1 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: fsmap 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: mgrmap e13: x(active, since 81s) 2024-04-26T21:22:42.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:41 smithi097 ceph-mon[28127]: overall HEALTH_OK 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T21:22:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-26T21:22:43.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: Updating smithi143:/etc/ceph/ceph.conf 2024-04-26T21:22:43.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:43.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:42 smithi111 ceph-mon[31001]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:43.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.218 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: Updating smithi143:/etc/ceph/ceph.conf 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:43.219 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:42 smithi097 ceph-mon[28127]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: Updating smithi143:/etc/ceph/ceph.conf 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:22:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:42 smithi143 ceph-mon[31461]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.088+0000 7f7f4f800700 1 -- 172.21.15.97:0/1470557693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48103090 msgr2=0x7f7f48103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.088+0000 7f7f4f800700 1 --2- 172.21.15.97:0/1470557693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48103090 0x7f7f48103470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009910 tx=0x7f7f3c009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.088+0000 7f7f4f800700 1 -- 172.21.15.97:0/1470557693 shutdown_connections 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.088+0000 7f7f4f800700 1 --2- 172.21.15.97:0/1470557693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48103090 0x7f7f48103470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.088+0000 7f7f4f800700 1 -- 172.21.15.97:0/1470557693 >> 172.21.15.97:0/1470557693 conn(0x7f7f480fe960 msgr2=0x7f7f48100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.089+0000 7f7f4f800700 1 -- 172.21.15.97:0/1470557693 shutdown_connections 2024-04-26T21:22:44.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.089+0000 7f7f4f800700 1 -- 172.21.15.97:0/1470557693 wait complete. 2024-04-26T21:22:44.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 Processor -- start 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 -- start start 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f48075c60 0x7f7f480760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d1c0 con 0x7f7f481a4c90 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d040 con 0x7f7f48103090 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.090+0000 7f7f4f800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d340 con 0x7f7f48075c60 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37410/0 (socket says 172.21.15.97:37410) 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 -- 172.21.15.97:0/2392277926 learned_addr learned my addr 172.21.15.97:0/2392277926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:22:44.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4ce00700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f48075c60 msgr2=0x7f7f480760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f48075c60 0x7f7f480760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4ce00700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 msgr2=0x7f7f48075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:22:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4ce00700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 msgr2=0x7f7f48075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 msgr2=0x7f7f48075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4ce00700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4ce00700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.091+0000 7f7f4d800700 1 -- 172.21.15.97:0/2392277926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f481a7750 con 0x7f7f481a4c90 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f4d800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000ba50 tx=0x7f7f4000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40019070 con 0x7f7f481a4c90 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f40014440 con 0x7f7f481a4c90 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400075b0 con 0x7f7f481a4c90 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f481a7a40 con 0x7f7f481a4c90 2024-04-26T21:22:44.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.092+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a7f30 con 0x7f7f481a4c90 2024-04-26T21:22:44.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.093+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7f4001c070 con 0x7f7f481a4c90 2024-04-26T21:22:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.093+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48064a80 con 0x7f7f481a4c90 2024-04-26T21:22:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.094+0000 7f7f45600700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 0x7f7f3003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.094+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7f4004bed0 con 0x7f7f481a4c90 2024-04-26T21:22:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.094+0000 7f7f4ce00700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 0x7f7f3003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.095+0000 7f7f4ce00700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 0x7f7f3003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006010 tx=0x7f7f3c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.100+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f4004b740 con 0x7f7f481a4c90 2024-04-26T21:22:44.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.339+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7f481a82c0 con 0x7f7f481a4c90 2024-04-26T21:22:44.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.340+0000 7f7f45600700 1 -- 172.21.15.97:0/2392277926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f7f400145f0 con 0x7f7f481a4c90 2024-04-26T21:22:44.340 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:44.340 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-04-26T21:22:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:22:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-04-26T21:22:44.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.342+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 msgr2=0x7f7f3003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:44.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.342+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 0x7f7f3003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006010 tx=0x7f7f3c01a040 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.342+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 msgr2=0x7f7f481a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.342+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000ba50 tx=0x7f7f4000be10 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 shutdown_connections 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7f30039ae0 0x7f7f3003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f48103090 0x7f7f48075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f481a4c90 0x7f7f481a7080 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 --2- 172.21.15.97:0/2392277926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f48075c60 0x7f7f480760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:44.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 >> 172.21.15.97:0/2392277926 conn(0x7f7f480fe960 msgr2=0x7f7f48104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:44.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 shutdown_connections 2024-04-26T21:22:44.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:44.343+0000 7f7f4f800700 1 -- 172.21.15.97:0/2392277926 wait complete. 2024-04-26T21:22:45.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: Updating smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2392277926' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:45.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:45.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: Updating smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2392277926' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:45.049 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.087 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:22:45.087 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:22:45.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:22:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: Updating smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: Updating smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: Updating smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2392277926' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:45.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:45.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:45.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:45.151 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:22:45.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:45.257 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:22:45.257 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:22:45.350 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:22:45.350 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:45.469 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:22:45.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:22:45.564 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:22:45.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:22:45.693 INFO:tasks.cephadm:Adding mgr.x on smithi097 2024-04-26T21:22:45.693 INFO:tasks.cephadm:Adding mgr.y on smithi111 2024-04-26T21:22:45.694 INFO:tasks.cephadm:Adding mgr.z on smithi143 2024-04-26T21:22:45.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi097=x;smithi111=y;smithi143=z' 2024-04-26T21:22:45.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:45 smithi097 ceph-mon[28127]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:22:45.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:45 smithi097 ceph-mon[28127]: Reconfiguring daemon mon.a on smithi097 2024-04-26T21:22:45.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:45 smithi097 ceph-mon[28127]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:45.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:45 smithi143 ceph-mon[31461]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:22:45.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:45 smithi143 ceph-mon[31461]: Reconfiguring daemon mon.a on smithi097 2024-04-26T21:22:45.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:45 smithi143 ceph-mon[31461]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:45 smithi111 ceph-mon[31001]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:22:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:45 smithi111 ceph-mon[31001]: Reconfiguring daemon mon.a on smithi097 2024-04-26T21:22:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:45 smithi111 ceph-mon[31001]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:22:47.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:22:47.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:47.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:47.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:47.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: Reconfiguring daemon mon.b on smithi111 2024-04-26T21:22:47.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:47 smithi143 ceph-mon[31461]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: Reconfiguring daemon mon.b on smithi111 2024-04-26T21:22:47.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:47 smithi111 ceph-mon[31001]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:47.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: Reconfiguring daemon mon.b on smithi111 2024-04-26T21:22:47.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:47 smithi097 ceph-mon[28127]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.133+0000 7fade5600700 1 -- 172.21.15.143:0/2841165742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 msgr2=0x7fade0101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.133+0000 7fade5600700 1 --2- 172.21.15.143:0/2841165742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0101c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009910 tx=0x7fadd0009c20 comp rx=0 tx=0).stop 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.134+0000 7fade5600700 1 -- 172.21.15.143:0/2841165742 shutdown_connections 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.134+0000 7fade5600700 1 --2- 172.21.15.143:0/2841165742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0101c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.134+0000 7fade5600700 1 -- 172.21.15.143:0/2841165742 >> 172.21.15.143:0/2841165742 conn(0x7fade00fd420 msgr2=0x7fade00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.134+0000 7fade5600700 1 -- 172.21.15.143:0/2841165742 shutdown_connections 2024-04-26T21:22:48.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.134+0000 7fade5600700 1 -- 172.21.15.143:0/2841165742 wait complete. 2024-04-26T21:22:48.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.135+0000 7fade5600700 1 Processor -- start 2024-04-26T21:22:48.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.135+0000 7fade5600700 1 -- start start 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0107380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fade01078c0 0x7fade0103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fade010af10 con 0x7fade0101800 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fade010ad90 con 0x7fade01078c0 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7fade5600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fade010b090 con 0x7fade0104320 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7faddf400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7faddf400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57658/0 (socket says 172.21.15.143:57658) 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7faddf400700 1 -- 172.21.15.143:0/2011536337 learned_addr learned my addr 172.21.15.143:0/2011536337 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.136+0000 7faddea00700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 -- 172.21.15.143:0/2011536337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fade01078c0 msgr2=0x7fade0103de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fade01078c0 0x7fade0103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 -- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 msgr2=0x7fade0107380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0107380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 -- 172.21.15.143:0/2011536337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01a71b0 con 0x7fade0104320 2024-04-26T21:22:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7faddf400700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadd800ba50 tx=0x7fadd800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8019070 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.137+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01a7380 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.138+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadd800ce40 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.138+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8007350 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.138+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a78e0 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.139+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade0064a80 con 0x7fade0104320 2024-04-26T21:22:48.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.139+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fadd801c070 con 0x7fade0104320 2024-04-26T21:22:48.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.139+0000 7fadd7400700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 0x7fadc403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:48.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.139+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fadd804bd60 con 0x7fade0104320 2024-04-26T21:22:48.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.140+0000 7faddea00700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 0x7fadc403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:48.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.141+0000 7faddea00700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 0x7fadc403bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009750 tx=0x7fadd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:48.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.145+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fadd8014680 con 0x7fade0104320 2024-04-26T21:22:48.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.390+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi111=y;smithi143=z", "target": ["mon-mgr", ""]}) -- 0x7fade00ff480 con 0x7fadc4039ae0 2024-04-26T21:22:48.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.405+0000 7fadd7400700 1 -- 172.21.15.143:0/2011536337 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fade00ff480 con 0x7fadc4039ae0 2024-04-26T21:22:48.407 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mgr update... 2024-04-26T21:22:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.407+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 msgr2=0x7fadc403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.407+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 0x7fadc403bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009750 tx=0x7fadd0005fb0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.407+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 msgr2=0x7fade01a6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.407+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadd800ba50 tx=0x7fadd800bd60 comp rx=0 tx=0).stop 2024-04-26T21:22:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 shutdown_connections 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fadc4039ae0 0x7fadc403bfa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fade0104320 0x7fade01a6c70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0101800 0x7fade0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 --2- 172.21.15.143:0/2011536337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fade01078c0 0x7fade0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 >> 172.21.15.143:0/2011536337 conn(0x7fade00fd420 msgr2=0x7fade00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 shutdown_connections 2024-04-26T21:22:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:22:48.408+0000 7fade5600700 1 -- 172.21.15.143:0/2011536337 wait complete. 2024-04-26T21:22:49.145 DEBUG:teuthology.orchestra.run.smithi111:mgr.y> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:22:49.153 DEBUG:teuthology.orchestra.run.smithi143:mgr.z> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:22:49.157 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:22:49.157 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:22:49.157 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:22:49.214 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:22:49.214 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-26T21:22:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:22:49.264 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 21:21:36.735200330 +0000 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 21:15:59.730342858 +0000 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 21:15:59.730342858 +0000 2024-04-26T21:22:49.265 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 21:15:59.730342858 +0000 2024-04-26T21:22:49.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:22:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T21:22:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T21:22:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000395096 s, 1.3 MB/s 2024-04-26T21:22:49.343 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:22:49.398 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-26T21:22:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:22:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:49.447 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T21:22:49.447 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:49.447 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:49.448 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 21:21:36.798199271 +0000 2024-04-26T21:22:49.448 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 21:16:00.320333376 +0000 2024-04-26T21:22:49.448 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 21:16:00.320333376 +0000 2024-04-26T21:22:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 21:16:00.320333376 +0000 2024-04-26T21:22:49.448 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:22:49.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T21:22:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T21:22:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000430953 s, 1.2 MB/s 2024-04-26T21:22:49.526 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:22:49.581 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-26T21:22:49.615 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi111=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: Saving service mgr spec with placement smithi097=x;smithi111=y;smithi143=z;count:3 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:49 smithi097 ceph-mon[28127]: Reconfiguring daemon mon.c on smithi143 2024-04-26T21:22:49.629 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 21:21:36.865198145 +0000 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 21:16:00.918323766 +0000 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 21:16:00.919323750 +0000 2024-04-26T21:22:49.630 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 21:16:00.918323766 +0000 2024-04-26T21:22:49.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi111=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: Saving service mgr spec with placement smithi097=x;smithi111=y;smithi143=z;count:3 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:22:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:49 smithi111 ceph-mon[31001]: Reconfiguring daemon mon.c on smithi143 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi111=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: Saving service mgr spec with placement smithi097=x;smithi111=y;smithi143=z;count:3 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:22:49.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:22:49.672 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:49.672 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:49 smithi143 ceph-mon[31461]: Reconfiguring daemon mon.c on smithi143 2024-04-26T21:22:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T21:22:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T21:22:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000413685 s, 1.2 MB/s 2024-04-26T21:22:49.756 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:22:49.809 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-26T21:22:49.860 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:22:49.860 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:49.860 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T21:22:49.860 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:49.861 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:49.861 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 21:21:36.926197120 +0000 2024-04-26T21:22:49.861 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 21:16:01.494314509 +0000 2024-04-26T21:22:49.861 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 21:16:01.494314509 +0000 2024-04-26T21:22:49.861 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 21:16:01.494314509 +0000 2024-04-26T21:22:49.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:22:49.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T21:22:49.936 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T21:22:49.936 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00034006 s, 1.5 MB/s 2024-04-26T21:22:49.940 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:22:49.993 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-26T21:22:49.993 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:22:50.051 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:22:50.051 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-04-26T21:22:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:22:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-26 21:22:33.549160248 +0000 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-26 21:16:01.934221742 +0000 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-26 21:16:01.934221742 +0000 2024-04-26T21:22:50.105 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-26 21:16:01.934221742 +0000 2024-04-26T21:22:50.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:22:50.184 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-26T21:22:50.184 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-26T21:22:50.184 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000374168 s, 1.4 MB/s 2024-04-26T21:22:50.188 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:22:50.243 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-26 21:22:33.550160230 +0000 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-26 21:16:02.481212264 +0000 2024-04-26T21:22:50.292 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-26 21:16:02.481212264 +0000 2024-04-26T21:22:50.293 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-26 21:16:02.481212264 +0000 2024-04-26T21:22:50.293 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:22:50.370 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-26T21:22:50.370 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-26T21:22:50.370 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000518392 s, 988 kB/s 2024-04-26T21:22:50.373 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:22:50.428 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-04-26T21:22:50.477 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:22:50.477 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 678 Links: 1 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-26 21:22:33.551160212 +0000 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-26 21:16:03.077201937 +0000 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-26 21:16:03.077201937 +0000 2024-04-26T21:22:50.478 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-26 21:16:03.077201937 +0000 2024-04-26T21:22:50.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:22:50.549 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-26T21:22:50.549 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-26T21:22:50.549 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.00033482 s, 1.5 MB/s 2024-04-26T21:22:50.552 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:22:50.603 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-26 21:22:33.551160212 +0000 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-26 21:16:03.677191540 +0000 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-26 21:16:03.678191523 +0000 2024-04-26T21:22:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-26 21:16:03.677191540 +0000 2024-04-26T21:22:50.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:22:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-26T21:22:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-26T21:22:50.762 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000412389 s, 1.2 MB/s 2024-04-26T21:22:50.765 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:22:50.817 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-04-26T21:22:50.817 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:22:50.862 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:22:50.862 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-04-26T21:22:50.947 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:22:50.947 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:50.947 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-04-26 21:22:22.395541347 +0000 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-04-26 21:16:01.894350059 +0000 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-04-26 21:16:01.894350059 +0000 2024-04-26T21:22:50.948 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-04-26 21:16:01.894350059 +0000 2024-04-26T21:22:50.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:22:51.066 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-04-26T21:22:51.066 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-04-26T21:22:51.066 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000345347 s, 1.5 MB/s 2024-04-26T21:22:51.070 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:22:51.114 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-04-26 21:22:22.396541320 +0000 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-04-26 21:16:02.508333616 +0000 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-04-26 21:16:02.508333616 +0000 2024-04-26T21:22:51.204 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-04-26 21:16:02.508333616 +0000 2024-04-26T21:22:51.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:22:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-04-26T21:22:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-04-26T21:22:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000450292 s, 1.1 MB/s 2024-04-26T21:22:51.271 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:22:51.319 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-04-26 21:22:22.397541293 +0000 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-04-26 21:16:03.095317895 +0000 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-04-26 21:16:03.095317895 +0000 2024-04-26T21:22:51.363 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-04-26 21:16:03.095317895 +0000 2024-04-26T21:22:51.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:22:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-04-26T21:22:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-04-26T21:22:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000294122 s, 1.7 MB/s 2024-04-26T21:22:51.468 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:22:51.511 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-04-26T21:22:51.593 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:22:51.593 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:22:51.593 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-04-26T21:22:51.593 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:22:51.593 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:22:51.594 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-04-26 21:22:22.397541293 +0000 2024-04-26T21:22:51.594 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-04-26 21:16:03.683302148 +0000 2024-04-26T21:22:51.594 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-04-26 21:16:03.683302148 +0000 2024-04-26T21:22:51.594 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-04-26 21:16:03.683302148 +0000 2024-04-26T21:22:51.594 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:22:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-04-26T21:22:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-04-26T21:22:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000346118 s, 1.5 MB/s 2024-04-26T21:22:51.708 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:22:51.759 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-26T21:22:51.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:51.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:51.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:51.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:51 smithi143 ceph-mon[31461]: Deploying daemon mgr.z on smithi143 2024-04-26T21:22:51.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:51.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:51.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:51 smithi111 ceph-mon[31001]: Deploying daemon mgr.z on smithi143 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:51 smithi097 ceph-mon[28127]: Deploying daemon mgr.z on smithi143 2024-04-26T21:22:52.190 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:53.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:53 smithi097 ceph-mon[28127]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:53.891 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:53 smithi143 ceph-mon[31461]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:53 smithi111 ceph-mon[31001]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:54.152 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 podman[33333]: 2024-04-26 21:22:54.150763876 +0000 UTC m=+0.387450926 container start 53d47b99af6e52dfc67be35a94f15064fa1ff2a7217e276757bfa15d704c3331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:22:54.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 ceph-mgr[33351]: -- 172.21.15.143:0/3863810377 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609371005a0 con 0x5609370d5000 2024-04-26T21:22:54.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 bash[33333]: 53d47b99af6e52dfc67be35a94f15064fa1ff2a7217e276757bfa15d704c3331 2024-04-26T21:22:54.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 systemd[1]: Started Ceph mgr.z for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:22:54.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:54.388+0000 7f0246b2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:22:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036171 s, 290 MB/s 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2024-04-26T21:22:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T21:22:55.045 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:54.648+0000 7f0246b2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:22:55.045 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:54 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:54.782+0000 7f0246b2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:55.181+0000 7f0246b2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:55.291+0000 7f0246b2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:55.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:55.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:55.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:55.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: Deploying daemon mgr.y on smithi111 2024-04-26T21:22:55.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:55 smithi143 ceph-mon[31461]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: Deploying daemon mgr.y on smithi111 2024-04-26T21:22:55.700 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:55 smithi097 ceph-mon[28127]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:55.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:22:55.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:55.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: Deploying daemon mgr.y on smithi111 2024-04-26T21:22:55.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:55 smithi111 ceph-mon[31001]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:22:57.295 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:56 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:56.852+0000 7f0246b2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:22:57.295 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:57 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:57.077+0000 7f0246b2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:22:57.569 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:57 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:57.295+0000 7f0246b2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:22:57.570 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 systemd[1]: Starting Ceph mgr.y for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:22:57.655 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:57 smithi097 ceph-mon[28127]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:57.863 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:57 smithi111 ceph-mon[31001]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:57.864 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 podman[32359]: 2024-04-26 21:22:57.564179545 +0000 UTC m=+0.032547355 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:22:57.864 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 podman[32359]: 2024-04-26 21:22:57.659316417 +0000 UTC m=+0.127684230 container create 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:22:57.864 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 podman[32359]: 2024-04-26 21:22:57.862791385 +0000 UTC m=+0.331159271 container init 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:22:58.042 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:57 smithi143 ceph-mon[31461]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:58.043 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:57 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:57.668+0000 7f0246b2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:22:58.043 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:57 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:57.789+0000 7f0246b2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:22:58.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 podman[32359]: 2024-04-26 21:22:57.87355775 +0000 UTC m=+0.341925561 container start 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:22:58.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 ceph-mgr[32377]: -- 172.21.15.111:0/3168830226 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d18125c5a0 con 0x55d181230800 2024-04-26T21:22:58.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 bash[32359]: 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 2024-04-26T21:22:58.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:57 smithi111 systemd[1]: Started Ceph mgr.y for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:22:58.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:22:58.111+0000 7f22a46b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:22:58.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 -- 172.21.15.97:0/958477922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 msgr2=0x7f4480076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:58.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 --2- 172.21.15.97:0/958477922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f4480076300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f446c009990 tx=0x7f446c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 -- 172.21.15.97:0/958477922 shutdown_connections 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 --2- 172.21.15.97:0/958477922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f4480111e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 --2- 172.21.15.97:0/958477922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 0x7f44800758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 --2- 172.21.15.97:0/958477922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f4480076300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 -- 172.21.15.97:0/958477922 >> 172.21.15.97:0/958477922 conn(0x7f44800fea30 msgr2=0x7f4480100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 -- 172.21.15.97:0/958477922 shutdown_connections 2024-04-26T21:22:58.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.148+0000 7f4487c00700 1 -- 172.21.15.97:0/958477922 wait complete. 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.149+0000 7f4487c00700 1 Processor -- start 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 -- start start 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 0x7f44801a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f44801a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44801154c0 con 0x7f4480075e80 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4480115340 con 0x7f448010fa70 2024-04-26T21:22:58.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4487c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4480115640 con 0x7f4480077450 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4485200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4485200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52834/0 (socket says 172.21.15.97:52834) 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.150+0000 7f4485200700 1 -- 172.21.15.97:0/2773709898 learned_addr learned my addr 172.21.15.97:0/2773709898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f44801a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f447fe00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 0x7f44801a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:58.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 -- 172.21.15.97:0/2773709898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 msgr2=0x7f44801a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 0x7f44801a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 -- 172.21.15.97:0/2773709898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 msgr2=0x7f44801a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f44801a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 -- 172.21.15.97:0/2773709898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44801a7810 con 0x7f4480075e80 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f44801a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:22:58.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.151+0000 7f4485200700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f447400d980 tx=0x7f447400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.152+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474017070 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.152+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4474005020 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.152+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44801a7b00 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.152+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474009a20 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.152+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44801a8040 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.154+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4474009bc0 con 0x7f4480075e80 2024-04-26T21:22:58.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.154+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448006a110 con 0x7f4480075e80 2024-04-26T21:22:58.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.154+0000 7f447d600700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 0x7f446803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:22:58.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.154+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f447404c330 con 0x7f4480075e80 2024-04-26T21:22:58.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.154+0000 7f447fe00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 0x7f446803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:22:58.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.155+0000 7f447fe00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 0x7f446803bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:22:58.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.160+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f44740054f0 con 0x7f4480075e80 2024-04-26T21:22:58.295 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:58 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:58.041+0000 7f0246b2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:22:58.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:22:58.403+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4480063860 con 0x7f4468039890 2024-04-26T21:22:58.642 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:22:58.367+0000 7f22a46b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:22:58.642 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:22:58.507+0000 7f22a46b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:22:58.794 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:58 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:58.295+0000 7f0246b2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:22:58.924+0000 7f22a46b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:22:59.039+0000 7f22a46b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:22:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:22:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:22:59 smithi111 ceph-mon[31001]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:59.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:59.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:22:59 smithi097 ceph-mon[28127]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:22:59.544 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:59.109+0000 7f0246b2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:22:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:59.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:59.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-mon[31461]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:00.044 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:59.684+0000 7f0246b2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:23:00.045 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:22:59 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:22:59.819+0000 7f0246b2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:23:00.544 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:00.085+0000 7f0246b2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:23:01.044 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:00.743+0000 7f0246b2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:23:01.045 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:00.865+0000 7f0246b2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:23:01.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:00 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:00.696+0000 7f22a46b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:23:01.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:00 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:00.932+0000 7f22a46b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.477 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:01.176+0000 7f0246b2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:23:01.477 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:01.351+0000 7f0246b2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:01.476+0000 7f0246b2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:01.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:01.534 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:01.535 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:01.149+0000 7f22a46b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:23:01.535 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:01.532+0000 7f22a46b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:23:01.794 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:01.625+0000 7f0246b2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:23:01.892 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:01.655+0000 7f22a46b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:23:02.294 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:01.999+0000 7f0246b2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:23:02.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:01 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:01.910+0000 7f22a46b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:23:02.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:02.159+0000 7f22a46b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:02.745+0000 7f0246b2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.747 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]: dispatch 2024-04-26T21:23:02.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]': finished 2024-04-26T21:23:02.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:23:02.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]: dispatch 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]': finished 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:23:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:02.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]: dispatch 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2781219552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b100e5ac-c305-4e0b-80d1-246e9107f082"}]': finished 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:23:02.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:03.295 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z[33347]: 2024-04-26T21:23:03.032+0000 7f0246b2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:23:03.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:02.952+0000 7f22a46b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:23:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:03.545+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f4474010d00 con 0x7f4480075e80 2024-04-26T21:23:03.676 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: Reconfiguring daemon mgr.x on smithi097 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: Standby manager daemon z started 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:03.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:03 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/698088873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:03.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: Reconfiguring daemon mgr.x on smithi097 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: Standby manager daemon z started 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:03.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:03 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/698088873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:03.884 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:23:03.885 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: Reconfiguring daemon mgr.x on smithi097 2024-04-26T21:23:03.885 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:03.885 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: Standby manager daemon z started 2024-04-26T21:23:03.885 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:23:03.886 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:03.886 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:23:03.886 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.143:0/3225390003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:03.886 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/698088873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:03.887 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:03.494+0000 7f22a46b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:23:03.887 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:03.623+0000 7f22a46b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:23:03.887 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:03 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:03.882+0000 7f22a46b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:23:04.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:04 smithi143 ceph-mon[31461]: mgrmap e14: x(active, since 103s), standbys: z 2024-04-26T21:23:04.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:23:04.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:04 smithi097 ceph-mon[28127]: mgrmap e14: x(active, since 103s), standbys: z 2024-04-26T21:23:04.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:23:04.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:04 smithi111 ceph-mon[31001]: mgrmap e14: x(active, since 103s), standbys: z 2024-04-26T21:23:04.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:23:04.892 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:04 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:04.534+0000 7f22a46b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:23:04.892 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:04 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:04.655+0000 7f22a46b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:23:05.270 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:04 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:04.970+0000 7f22a46b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:23:05.270 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:05 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:05.144+0000 7f22a46b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:23:05.270 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:05 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:05.268+0000 7f22a46b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:23:05.525 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:05 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:05.417+0000 7f22a46b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:23:05.782 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:05 smithi111 ceph-mon[31001]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:05.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:05 smithi143 ceph-mon[31461]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:05.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:05 smithi097 ceph-mon[28127]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:06.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:05 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:05.780+0000 7f22a46b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:23:06.803 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:06.522+0000 7f22a46b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:23:07.142 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:23:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y[32373]: 2024-04-26T21:23:06.801+0000 7f22a46b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:23:07.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:07.551+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f44740051d0 con 0x7f4480075e80 2024-04-26T21:23:07.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:07.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:23:07.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: Standby manager daemon y started 2024-04-26T21:23:07.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:07.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:23:07.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:07 smithi097 ceph-mon[28127]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:07.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:07.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:23:07.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: Standby manager daemon y started 2024-04-26T21:23:07.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:07.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:23:07.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:07 smithi111 ceph-mon[31001]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: Standby manager daemon y started 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:23:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:07 smithi143 ceph-mon[31461]: from='mgr.? 172.21.15.111:0/1355650575' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:23:08.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:08 smithi097 ceph-mon[28127]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-04-26T21:23:08.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:08 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:23:08.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:08 smithi111 ceph-mon[31001]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-04-26T21:23:08.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:08 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:23:09.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:08 smithi143 ceph-mon[31461]: mgrmap e15: x(active, since 107s), standbys: z, y 2024-04-26T21:23:09.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:08 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:23:09.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:09 smithi097 ceph-mon[28127]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:09.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:09.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:09.665 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:09.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:09 smithi111 ceph-mon[31001]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:09.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:09 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:09.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:09 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:09.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:09 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:09 smithi143 ceph-mon[31461]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:09 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:09 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:09 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:11.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:11 smithi111 ceph-mon[31001]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:11.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:11.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:11.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:11.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:11 smithi097 ceph-mon[28127]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:11.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:11.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:11.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:12.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:11 smithi143 ceph-mon[31461]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:12.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:23:12.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:23:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:13.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:23:13.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:13.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:13 smithi097 ceph-mon[28127]: Deploying daemon osd.0 on smithi097 2024-04-26T21:23:13.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:13 smithi097 ceph-mon[28127]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:13.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:13 smithi111 ceph-mon[31001]: Deploying daemon osd.0 on smithi097 2024-04-26T21:23:13.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:13 smithi111 ceph-mon[31001]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:14.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:13 smithi143 ceph-mon[31461]: Deploying daemon osd.0 on smithi097 2024-04-26T21:23:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:13 smithi143 ceph-mon[31461]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:15.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:15 smithi097 ceph-mon[28127]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:15.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:15 smithi111 ceph-mon[31001]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:16.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:15 smithi143 ceph-mon[31461]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:17.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:17 smithi097 ceph-mon[28127]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:17.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:17 smithi111 ceph-mon[31001]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:17 smithi143 ceph-mon[31461]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:19.781 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:19 smithi097 ceph-mon[28127]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:19 smithi111 ceph-mon[31001]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:19 smithi143 ceph-mon[31461]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:21.433 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.433 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:21.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:21 smithi097 ceph-mon[28127]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:21.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:21.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:21 smithi143 ceph-mon[31461]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:21.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:21.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:21.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:21 smithi111 ceph-mon[31001]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:23.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:23.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:23.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:23.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:24.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:24 smithi097 ceph-mon[28127]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:23:24.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:24 smithi111 ceph-mon[31001]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:23:24.964 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2024-04-26T21:23:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.962+0000 7f447d600700 1 -- 172.21.15.97:0/2773709898 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4480063860 con 0x7f4468039890 2024-04-26T21:23:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 msgr2=0x7f446803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 0x7f446803bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 msgr2=0x7f448019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f447400d980 tx=0x7f447400dc90 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 shutdown_connections 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4468039890 0x7f446803bd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f448010fa70 0x7f44801a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4480075e80 0x7f448019fbb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 --2- 172.21.15.97:0/2773709898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4480077450 0x7f44801a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 >> 172.21.15.97:0/2773709898 conn(0x7f44800fea30 msgr2=0x7f4480102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 shutdown_connections 2024-04-26T21:23:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:23:24.966+0000 7f4487c00700 1 -- 172.21.15.97:0/2773709898 wait complete. 2024-04-26T21:23:25.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:24 smithi143 ceph-mon[31461]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:25.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:25.785 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:23:25.790 INFO:tasks.cephadm:Deploying osd.1 on smithi111 with /dev/vg_nvme/lv_4... 2024-04-26T21:23:25.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:25.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:25.871 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:26.223 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:23:26.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:26 smithi111 ceph-mon[31001]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:23:26.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:26 smithi111 ceph-mon[31001]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:23:26.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:26 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:26.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:26 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:26 smithi143 ceph-mon[31461]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:23:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:26 smithi143 ceph-mon[31461]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:23:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:26 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:26 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-mon[28127]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:23:27.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-mon[28127]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:23:27.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.053 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0[43242]: 2024-04-26T21:23:26.602+0000 7fb97f000700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:23:27.053 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:23:26 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0[43242]: 2024-04-26T21:23:26.648+0000 7fb974e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:23:27.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:27 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:23:27.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:27 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:23:27.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:27 smithi111 ceph-mon[31001]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:27.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:27 smithi111 ceph-mon[31001]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' 2024-04-26T21:23:27.736 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:27 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:23:27.736 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:27 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:23:27.736 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:27 smithi097 ceph-mon[28127]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:27.736 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:27.736 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:27 smithi097 ceph-mon[28127]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' 2024-04-26T21:23:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:27 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:23:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:27 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:23:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:27 smithi143 ceph-mon[31461]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:27 smithi143 ceph-mon[31461]: from='osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768]' entity='osd.0' 2024-04-26T21:23:28.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:28.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:28 smithi111 ceph-mon[31001]: osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] boot 2024-04-26T21:23:28.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:28 smithi111 ceph-mon[31001]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:23:28.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0351854 s, 298 MB/s 2024-04-26T21:23:28.914 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-26T21:23:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:28 smithi143 ceph-mon[31461]: osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] boot 2024-04-26T21:23:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:28 smithi143 ceph-mon[31461]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:23:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:28 smithi097 ceph-mon[28127]: osd.0 [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] boot 2024-04-26T21:23:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:28 smithi097 ceph-mon[28127]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:23:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:23:29.692 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-04-26T21:23:29.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:29 smithi097 ceph-mon[28127]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:29.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:29 smithi097 ceph-mon[28127]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:23:29.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:29 smithi111 ceph-mon[31001]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:29.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:29 smithi111 ceph-mon[31001]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:23:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:29 smithi143 ceph-mon[31461]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:23:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:29 smithi143 ceph-mon[31461]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:23:30.086 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi097 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi097 to 17089M 2024-04-26T21:23:31.474 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.475 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:31.475 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:31.475 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.475 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:31 smithi111 ceph-mon[31001]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi097 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi097 to 17089M 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:31.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:31.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:31 smithi143 ceph-mon[31461]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:31.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi097 2024-04-26T21:23:31.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi097 to 17089M 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:31.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:31 smithi097 ceph-mon[28127]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 -- 172.21.15.111:0/2306971172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac101d00 msgr2=0x7f64ac104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 --2- 172.21.15.111:0/2306971172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac101d00 0x7f64ac104130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f649c009990 tx=0x7f649c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 -- 172.21.15.111:0/2306971172 shutdown_connections 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 --2- 172.21.15.111:0/2306971172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac101d00 0x7f64ac104130 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 --2- 172.21.15.111:0/2306971172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 0x7f64ac1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 --2- 172.21.15.111:0/2306971172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac104670 0x7f64ac106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.249+0000 7f64b0e00700 1 -- 172.21.15.111:0/2306971172 >> 172.21.15.111:0/2306971172 conn(0x7f64ac0fd920 msgr2=0x7f64ac0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.250+0000 7f64b0e00700 1 -- 172.21.15.111:0/2306971172 shutdown_connections 2024-04-26T21:23:32.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.250+0000 7f64b0e00700 1 -- 172.21.15.111:0/2306971172 wait complete. 2024-04-26T21:23:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.251+0000 7f64b0e00700 1 Processor -- start 2024-04-26T21:23:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.251+0000 7f64b0e00700 1 -- start start 2024-04-26T21:23:32.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac101d00 0x7f64ac19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 0x7f64ac1a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac115340 con 0x7f64ac101d00 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1151c0 con 0x7f64ac104670 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64b0e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1154c0 con 0x7f64ac107040 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64a9400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64a9400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47182/0 (socket says 172.21.15.111:47182) 2024-04-26T21:23:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.252+0000 7f64a9400700 1 -- 172.21.15.111:0/505763893 learned_addr learned my addr 172.21.15.111:0/505763893 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-26T21:23:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac101d00 0x7f64ac19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64aa800700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 0x7f64ac1a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 -- 172.21.15.111:0/505763893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 msgr2=0x7f64ac1a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 0x7f64ac1a7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 -- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac101d00 msgr2=0x7f64ac19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac101d00 0x7f64ac19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 -- 172.21.15.111:0/505763893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac1a76f0 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.253+0000 7f64a9400700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f649800d950 tx=0x7f649800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.254+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498016070 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.254+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f649800fd20 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.254+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac1a79e0 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.254+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649800e3b0 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.254+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1a7f70 con 0x7f64ac104670 2024-04-26T21:23:32.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.255+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac06a110 con 0x7f64ac104670 2024-04-26T21:23:32.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.259+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6498019070 con 0x7f64ac104670 2024-04-26T21:23:32.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.260+0000 7f64a2a00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 0x7f64900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:23:32.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.260+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f64980ce370 con 0x7f64ac104670 2024-04-26T21:23:32.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.261+0000 7f64a9e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 0x7f64900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:23:32.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.262+0000 7f64a9e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 0x7f64900a60f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f649c006010 tx=0x7f649c0037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:23:32.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.262+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f649809c720 con 0x7f64ac104670 2024-04-26T21:23:32.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:32.499+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f64ac063860 con 0x7f64900a3c30 2024-04-26T21:23:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:23:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:23:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:23:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:23:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:23:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:23:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:33.779 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:33 smithi111 ceph-mon[31001]: from='client.34133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:23:33.779 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:33 smithi111 ceph-mon[31001]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:33 smithi143 ceph-mon[31461]: from='client.34133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:23:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:33 smithi143 ceph-mon[31461]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:34.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:33 smithi097 ceph-mon[28127]: from='client.34133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:23:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:33 smithi097 ceph-mon[28127]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:35 smithi111 ceph-mon[31001]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:35 smithi143 ceph-mon[31461]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:35 smithi097 ceph-mon[28127]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:36 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/1973178636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:36 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:36 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]': finished 2024-04-26T21:23:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:36 smithi111 ceph-mon[31001]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:23:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:23:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:36 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/1973178636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:36 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:36 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]': finished 2024-04-26T21:23:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:36 smithi143 ceph-mon[31461]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:23:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:23:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/1973178636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:36 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]: dispatch 2024-04-26T21:23:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:36 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2702d8b6-9f28-421d-b2d1-9c38a9900bbf"}]': finished 2024-04-26T21:23:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:36 smithi097 ceph-mon[28127]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:23:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:23:37.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:37 smithi111 ceph-mon[31001]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:37.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:37 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/1755141060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:37 smithi143 ceph-mon[31461]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:37 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/1755141060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:37 smithi097 ceph-mon[28127]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:37 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/1755141060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:23:39.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:39 smithi111 ceph-mon[31001]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:39 smithi143 ceph-mon[31461]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:39 smithi097 ceph-mon[28127]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:41.646 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:41 smithi111 ceph-mon[31001]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:41 smithi143 ceph-mon[31461]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:41 smithi097 ceph-mon[28127]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:43.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:43 smithi111 ceph-mon[31001]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:44.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:43 smithi143 ceph-mon[31461]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:43 smithi097 ceph-mon[28127]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:45.879 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:45 smithi111 ceph-mon[31001]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:45 smithi143 ceph-mon[31461]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:45 smithi097 ceph-mon[28127]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:46.947 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:23:46.947 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:47.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:23:47.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:23:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:47.934 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:47 smithi111 ceph-mon[31001]: Deploying daemon osd.1 on smithi111 2024-04-26T21:23:47.934 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:47 smithi111 ceph-mon[31001]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:47 smithi143 ceph-mon[31461]: Deploying daemon osd.1 on smithi111 2024-04-26T21:23:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:47 smithi143 ceph-mon[31461]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:47 smithi097 ceph-mon[28127]: Deploying daemon osd.1 on smithi111 2024-04-26T21:23:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:47 smithi097 ceph-mon[28127]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:49 smithi143 ceph-mon[31461]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:50.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:49 smithi097 ceph-mon[28127]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:49 smithi111 ceph-mon[31001]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:51.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:51 smithi097 ceph-mon[28127]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:51.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:51 smithi111 ceph-mon[31001]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:51 smithi143 ceph-mon[31461]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:53 smithi143 ceph-mon[31461]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:53 smithi097 ceph-mon[28127]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:54.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:53 smithi111 ceph-mon[31001]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:54 smithi111 ceph-mon[31001]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:54 smithi143 ceph-mon[31461]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:55.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:55.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:55.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:54 smithi097 ceph-mon[28127]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:57.866 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:57.866 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:57.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:57.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:57.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:57.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:58.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:58.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.526+0000 7f64a2a00700 1 -- 172.21.15.111:0/505763893 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f64ac063860 con 0x7f64900a3c30 2024-04-26T21:23:58.528 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 1 on host 'smithi111' 2024-04-26T21:23:58.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 msgr2=0x7f64900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 0x7f64900a60f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f649c006010 tx=0x7f649c0037c0 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 msgr2=0x7f64ac19ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f649800d950 tx=0x7f649800dc60 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 shutdown_connections 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f64900a3c30 0x7f64900a60f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64ac107040 0x7f64ac1a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac101d00 0x7f64ac19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 --2- 172.21.15.111:0/505763893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64ac104670 0x7f64ac19ffd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 >> 172.21.15.111:0/505763893 conn(0x7f64ac0fd920 msgr2=0x7f64ac105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 shutdown_connections 2024-04-26T21:23:58.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:23:58.530+0000 7f64b0e00700 1 -- 172.21.15.111:0/505763893 wait complete. 2024-04-26T21:23:58.876 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:58 smithi111 ceph-mon[31001]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:58.876 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:58 smithi111 ceph-mon[31001]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:58.876 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:58 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.877 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:58 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:58.877 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:58 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:58 smithi143 ceph-mon[31461]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:58 smithi143 ceph-mon[31461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:58 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:58 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:58 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:58 smithi097 ceph-mon[28127]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:58 smithi097 ceph-mon[28127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:23:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:23:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:58 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:59.275 DEBUG:teuthology.orchestra.run.smithi111:osd.1> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:23:59.278 INFO:tasks.cephadm:Deploying osd.2 on smithi143 with /dev/vg_nvme/lv_4... 2024-04-26T21:23:59.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:23:59.719 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:23:59.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:23:59 smithi143 ceph-mon[31461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:23:59.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:23:59.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:23:59 smithi111 ceph-mon[31001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: from='osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:24:00.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:23:59 smithi097 ceph-mon[28127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:24:00.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:00 smithi111 ceph-mon[31001]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:24:00.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:00 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:00 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.862 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:00 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1[35583]: 2024-04-26T21:24:00.642+0000 7f5f3f400700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:24:00.862 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:00 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1[35583]: 2024-04-26T21:24:00.689+0000 7f5f35200700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:24:00.930 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:00 smithi097 ceph-mon[28127]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:24:00.931 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:00 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.931 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:00 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:00 smithi143 ceph-mon[31461]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:00 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:00 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:01.784 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:01 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:24:01.785 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:01 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:24:01.785 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:01 smithi143 ceph-mon[31461]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:01.785 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:01.785 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:01 smithi143 ceph-mon[31461]: from='osd.1 ' entity='osd.1' 2024-04-26T21:24:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:01 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:24:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:01 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:24:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:01 smithi111 ceph-mon[31001]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:01 smithi111 ceph-mon[31001]: from='osd.1 ' entity='osd.1' 2024-04-26T21:24:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:01 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:24:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:01 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:24:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:01 smithi097 ceph-mon[28127]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:01 smithi097 ceph-mon[28127]: from='osd.1 ' entity='osd.1' 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0361671 s, 290 MB/s 2024-04-26T21:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-04-26T21:24:02.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:02.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:02 smithi111 ceph-mon[31001]: osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] boot 2024-04-26T21:24:02.868 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:02 smithi111 ceph-mon[31001]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:24:02.868 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:02 smithi143 ceph-mon[31461]: osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] boot 2024-04-26T21:24:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:02 smithi143 ceph-mon[31461]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:24:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:03.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:02 smithi097 ceph-mon[28127]: osd.1 [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] boot 2024-04-26T21:24:03.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:02 smithi097 ceph-mon[28127]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:24:03.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:24:03.488 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-04-26T21:24:03.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:03.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:03.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:03.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:03.915 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:05.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:04 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi111 2024-04-26T21:24:05.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:04 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-26T21:24:05.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:05.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:04 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi111 2024-04-26T21:24:05.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:04 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-26T21:24:05.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:04 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi111 2024-04-26T21:24:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:04 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi111 to 17089M 2024-04-26T21:24:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 -- 172.21.15.143:0/1240897725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092410acc0 msgr2=0x7f092410b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 --2- 172.21.15.143:0/1240897725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092410acc0 0x7f092410b120 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0910009990 tx=0x7f0910009ca0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 -- 172.21.15.143:0/1240897725 shutdown_connections 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 --2- 172.21.15.143:0/1240897725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410b660 0x7f0924111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 --2- 172.21.15.143:0/1240897725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924109ac0 0x7f0924109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 --2- 172.21.15.143:0/1240897725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092410acc0 0x7f092410b120 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 -- 172.21.15.143:0/1240897725 >> 172.21.15.143:0/1240897725 conn(0x7f0924078a90 msgr2=0x7f092407aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 -- 172.21.15.143:0/1240897725 shutdown_connections 2024-04-26T21:24:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.882+0000 7f0928a00700 1 -- 172.21.15.143:0/1240897725 wait complete. 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.883+0000 7f0928a00700 1 Processor -- start 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.883+0000 7f0928a00700 1 -- start start 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 0x7f09241a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f092410b660 0x7f09241a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115410 con 0x7f092410b660 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115290 con 0x7f092410acc0 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0928a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115590 con 0x7f0924109ac0 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0921e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0921e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45228/0 (socket says 172.21.15.143:45228) 2024-04-26T21:24:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.884+0000 7f0921e00700 1 -- 172.21.15.143:0/3289262326 learned_addr learned my addr 172.21.15.143:0/3289262326 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921400700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 0x7f09241a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0922800700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f092410b660 0x7f09241a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 -- 172.21.15.143:0/3289262326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 msgr2=0x7f09241a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 0x7f09241a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 -- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f092410b660 msgr2=0x7f09241a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f092410b660 0x7f09241a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 -- 172.21.15.143:0/3289262326 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241a7740 con 0x7f0924109ac0 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921400700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 0x7f09241a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:24:05.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f0921e00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f091400d9b0 tx=0x7f091400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.885+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914017030 con 0x7f0924109ac0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.886+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09140044b0 con 0x7f0924109ac0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.886+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09241a7a30 con 0x7f0924109ac0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.886+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140038e0 con 0x7f0924109ac0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.886+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09241a7f70 con 0x7f0924109ac0 2024-04-26T21:24:05.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.887+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092406a110 con 0x7f0924109ac0 2024-04-26T21:24:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.891+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f091401a070 con 0x7f0924109ac0 2024-04-26T21:24:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.892+0000 7f091aa00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 0x7f09080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.893+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f09140ce9a0 con 0x7f0924109ac0 2024-04-26T21:24:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.893+0000 7f0921400700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 0x7f09080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.894+0000 7f0921400700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 0x7f09080a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091000b5c0 tx=0x7f0910003820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:05.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:05.895+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f091409cb30 con 0x7f0924109ac0 2024-04-26T21:24:06.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:05 smithi143 ceph-mon[31461]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:06.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:05 smithi097 ceph-mon[28127]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:06.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:06.113+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0924063860 con 0x7f09080a3bb0 2024-04-26T21:24:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:05 smithi111 ceph-mon[31001]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:07.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:07.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:07.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:07.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:07.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:07.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:07 smithi143 ceph-mon[31461]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:07 smithi143 ceph-mon[31461]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:07 smithi097 ceph-mon[28127]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:07 smithi097 ceph-mon[28127]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:07 smithi111 ceph-mon[31001]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:07 smithi111 ceph-mon[31001]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:09 smithi143 ceph-mon[31461]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:10.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:09 smithi097 ceph-mon[28127]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:09 smithi111 ceph-mon[31001]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:10 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/2650252531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:10 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:10 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]': finished 2024-04-26T21:24:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:10 smithi143 ceph-mon[31461]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:24:11.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:10 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/2650252531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:10 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:10 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]': finished 2024-04-26T21:24:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:10 smithi097 ceph-mon[28127]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:24:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:11.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:10 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/2650252531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:10 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]: dispatch 2024-04-26T21:24:11.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:10 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7059e770-39f5-4850-bfa3-521f33c394d0"}]': finished 2024-04-26T21:24:11.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:10 smithi111 ceph-mon[31001]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:24:11.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:11.850 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:11 smithi111 ceph-mon[31001]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:11.850 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:11 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/4060081018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:12.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:11 smithi143 ceph-mon[31461]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:11 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/4060081018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:12.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:11 smithi097 ceph-mon[28127]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:11 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/4060081018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:13 smithi143 ceph-mon[31461]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:14.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:13 smithi097 ceph-mon[28127]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:13 smithi111 ceph-mon[31001]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:15.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:15 smithi143 ceph-mon[31461]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:15 smithi097 ceph-mon[28127]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:15 smithi111 ceph-mon[31001]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:18.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:17 smithi143 ceph-mon[31461]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:18.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:17 smithi097 ceph-mon[28127]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:17 smithi111 ceph-mon[31001]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:19 smithi143 ceph-mon[31461]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:19 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:24:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:19 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:19 smithi097 ceph-mon[28127]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:19 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:24:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:19 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:19 smithi111 ceph-mon[31001]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:19 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:24:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:19 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:21.026 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:20 smithi143 ceph-mon[31461]: Deploying daemon osd.2 on smithi143 2024-04-26T21:24:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:20 smithi097 ceph-mon[28127]: Deploying daemon osd.2 on smithi143 2024-04-26T21:24:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:20 smithi111 ceph-mon[31001]: Deploying daemon osd.2 on smithi143 2024-04-26T21:24:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:21 smithi143 ceph-mon[31461]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:22.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:21 smithi097 ceph-mon[28127]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:21 smithi111 ceph-mon[31001]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:23.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:22 smithi097 ceph-mon[28127]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:22 smithi111 ceph-mon[31001]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:23.229 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:22 smithi143 ceph-mon[31461]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:25.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:25 smithi111 ceph-mon[31001]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:25 smithi143 ceph-mon[31461]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:26.132 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:25 smithi097 ceph-mon[28127]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:27.630 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:27 smithi143 ceph-mon[31461]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:27 smithi111 ceph-mon[31001]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:27 smithi097 ceph-mon[28127]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:28 smithi143 ceph-mon[31461]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:28 smithi097 ceph-mon[28127]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:28 smithi111 ceph-mon[31001]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:31.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:31 smithi143 ceph-mon[31461]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:31.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:31.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:31.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:31 smithi111 ceph-mon[31001]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:31.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:31.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:31 smithi097 ceph-mon[28127]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.536 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 2 on host 'smithi143' 2024-04-26T21:24:32.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.533+0000 7f091aa00700 1 -- 172.21.15.143:0/3289262326 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0924063860 con 0x7f09080a3bb0 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 msgr2=0x7f09080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 0x7f09080a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091000b5c0 tx=0x7f0910003820 comp rx=0 tx=0).stop 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 msgr2=0x7f092419fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f091400d9b0 tx=0x7f091400dd70 comp rx=0 tx=0).stop 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 shutdown_connections 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f09080a3bb0 0x7f09080a6070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f092410b660 0x7f09241a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:32.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0924109ac0 0x7f092419fae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:32.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 --2- 172.21.15.143:0/3289262326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f092410acc0 0x7f09241a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:32.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 >> 172.21.15.143:0/3289262326 conn(0x7f0924078a90 msgr2=0x7f092410e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:32.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.538+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 shutdown_connections 2024-04-26T21:24:32.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:24:32.539+0000 7f0928a00700 1 -- 172.21.15.143:0/3289262326 wait complete. 2024-04-26T21:24:32.783 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:32 smithi143 ceph-mon[31461]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:32.783 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:32 smithi143 ceph-mon[31461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:32.783 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.783 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:32 smithi111 ceph-mon[31001]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:32 smithi111 ceph-mon[31001]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:32.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:32.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:32 smithi097 ceph-mon[28127]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:32 smithi097 ceph-mon[28127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:24:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:33.264 DEBUG:teuthology.orchestra.run.smithi143:osd.2> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:24:33.268 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-26T21:24:33.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:24:33.701 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:33.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:33 smithi111 ceph-mon[31001]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:33.897 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:33.897 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:24:33.897 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:33.897 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:24:33.898 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:33.898 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:33 smithi143 ceph-mon[31461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:33.961 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:33.962 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:24:33.962 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: from='osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:33.962 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:24:33.962 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:33.962 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:33 smithi097 ceph-mon[28127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:24:34.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-mon[31461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:24:34.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-mon[31461]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:24:34.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:34.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:34.795 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2[36643]: 2024-04-26T21:24:34.621+0000 7f4ae7600700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:24:34.795 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:34 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2[36643]: 2024-04-26T21:24:34.689+0000 7f4add400700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:24:34.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:34 smithi097 ceph-mon[28127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:24:34.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:34 smithi097 ceph-mon[28127]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:24:34.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:34.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:34.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:34 smithi111 ceph-mon[31001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:24:34.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:34 smithi111 ceph-mon[31001]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:24:34.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:34.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:35 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:24:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:35 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:24:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:35 smithi097 ceph-mon[28127]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:35 smithi097 ceph-mon[28127]: from='osd.2 ' entity='osd.2' 2024-04-26T21:24:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:35 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:24:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:35 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:24:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:35 smithi111 ceph-mon[31001]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:35.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:35 smithi111 ceph-mon[31001]: from='osd.2 ' entity='osd.2' 2024-04-26T21:24:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:35 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:24:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:35 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:24:36.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:35 smithi143 ceph-mon[31461]: pgmap v104: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:36.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:35 smithi143 ceph-mon[31461]: from='osd.2 ' entity='osd.2' 2024-04-26T21:24:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:24:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:24:36.393 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T21:24:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T21:24:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0338116 s, 310 MB/s 2024-04-26T21:24:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T21:24:36.777 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.777 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:36 smithi097 ceph-mon[28127]: osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] boot 2024-04-26T21:24:36.777 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:36 smithi097 ceph-mon[28127]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:24:36.777 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.884 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.884 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:36 smithi143 ceph-mon[31461]: osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] boot 2024-04-26T21:24:36.884 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:36 smithi143 ceph-mon[31461]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:24:36.885 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:36 smithi111 ceph-mon[31001]: osd.2 [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] boot 2024-04-26T21:24:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:36 smithi111 ceph-mon[31001]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:24:36.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:24:37.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-26T21:24:37.560 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: pgmap v106: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:37.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:37 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:38.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: pgmap v106: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:24:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:37 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: pgmap v106: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:24:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:24:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:37 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:24:38.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi143 2024-04-26T21:24:38.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi143 to 17089M 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:38.923 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi143 2024-04-26T21:24:39.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi143 to 17089M 2024-04-26T21:24:39.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:39.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi143 2024-04-26T21:24:39.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi143 to 17089M 2024-04-26T21:24:39.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47691]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47691]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47696]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47696]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.201 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47696]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40483]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40483]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40483]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40483]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40486]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40486]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40486]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.620 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40486]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39526]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:39.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39526]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39526]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39529]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:39.643 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.643 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39529]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.643 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39529]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 -- 172.21.15.97:0/552298057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 msgr2=0x7fa9f810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/552298057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f810db10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000b0d0 tx=0x7fa9f000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 -- 172.21.15.97:0/552298057 shutdown_connections 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/552298057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 0x7fa9f8105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/552298057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f810db10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/552298057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 -- 172.21.15.97:0/552298057 >> 172.21.15.97:0/552298057 conn(0x7fa9f8100c80 msgr2=0x7fa9f81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 -- 172.21.15.97:0/552298057 shutdown_connections 2024-04-26T21:24:39.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.766+0000 7fa9fdc00700 1 -- 172.21.15.97:0/552298057 wait complete. 2024-04-26T21:24:39.768 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47719]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47719]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:39.769 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 sudo[47722]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:39.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 Processor -- start 2024-04-26T21:24:39.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 -- start start 2024-04-26T21:24:39.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 0x7fa9f819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:39.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:39.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8111090 con 0x7fa9f8107260 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8110f10 con 0x7fa9f81056c0 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.768+0000 7fa9fdc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8111210 con 0x7fa9f81068c0 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44536/0 (socket says 172.21.15.97:44536) 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 -- 172.21.15.97:0/3187524235 learned_addr learned my addr 172.21.15.97:0/3187524235 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f6a00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 0x7fa9f819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f6000700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 -- 172.21.15.97:0/3187524235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 msgr2=0x7fa9f819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f819bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 -- 172.21.15.97:0/3187524235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 msgr2=0x7fa9f819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 0x7fa9f819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:24:39.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f7400700 1 -- 172.21.15.97:0/3187524235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f81a3360 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.769+0000 7fa9f6000700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9f7400700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009c00 tx=0x7fa9f000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f001f030 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9f0007bc0 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f81a35f0 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f001b690 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.770+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f81a3a10 con 0x7fa9f8107260 2024-04-26T21:24:39.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.771+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f806a110 con 0x7fa9f8107260 2024-04-26T21:24:39.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.774+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa9f00055f0 con 0x7fa9f8107260 2024-04-26T21:24:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.775+0000 7fa9ef400700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:24:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.775+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7fa9f00d0330 con 0x7fa9f8107260 2024-04-26T21:24:39.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.775+0000 7fa9f6a00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:24:39.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.776+0000 7fa9f6a00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:24:39.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:39.779+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa9f009e250 con 0x7fa9f8107260 2024-04-26T21:24:40.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:40.019+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa9f8063860 con 0x7fa9dc0a3bb0 2024-04-26T21:24:40.021 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:40.021 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:40.021 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:24:40.021 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40489]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40489]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40489]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40489]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40492]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 sudo[40492]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.047 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39532]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39532]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39532]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39532]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39535]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39535]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39535]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 sudo[39535]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:24:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:40.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:24:40.821+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9f009a1c0 con 0x7fa9f8107260 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:40 smithi097 ceph-mon[28127]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:41.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:41.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:40 smithi111 ceph-mon[31001]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:24:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T21:24:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:41.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:40 smithi143 ceph-mon[31461]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:41 smithi097 ceph-mon[28127]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:24:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:41 smithi097 ceph-mon[28127]: Cluster is now healthy 2024-04-26T21:24:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:41 smithi097 ceph-mon[28127]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:24:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:41 smithi111 ceph-mon[31001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:24:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:41 smithi111 ceph-mon[31001]: Cluster is now healthy 2024-04-26T21:24:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:41 smithi111 ceph-mon[31001]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:24:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:41 smithi143 ceph-mon[31461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:24:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:41 smithi143 ceph-mon[31461]: Cluster is now healthy 2024-04-26T21:24:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:41 smithi143 ceph-mon[31461]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:24:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:42 smithi111 ceph-mon[31001]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:42 smithi097 ceph-mon[28127]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:43.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:42 smithi143 ceph-mon[31461]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:44.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:44 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]: dispatch 2024-04-26T21:24:44.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:44 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]': finished 2024-04-26T21:24:44.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:44 smithi097 ceph-mon[28127]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:24:44.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:24:44.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:44 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]: dispatch 2024-04-26T21:24:44.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:44 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]': finished 2024-04-26T21:24:44.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:44 smithi111 ceph-mon[31001]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:24:44.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:24:44.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:44 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]: dispatch 2024-04-26T21:24:44.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:44 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/884206991' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8d302e-f0d1-4f1d-9464-905160e2eb4d"}]': finished 2024-04-26T21:24:44.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:44 smithi143 ceph-mon[31461]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:24:44.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:24:45.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:45 smithi143 ceph-mon[31461]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:45.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:45 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1486051892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:45.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:45 smithi111 ceph-mon[31001]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:45.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:45 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1486051892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:45 smithi097 ceph-mon[28127]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:45 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1486051892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:24:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:47 smithi111 ceph-mon[31001]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:48.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:47 smithi143 ceph-mon[31461]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:47 smithi097 ceph-mon[28127]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:49.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:49 smithi097 ceph-mon[28127]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:49.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:49 smithi111 ceph-mon[31001]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:50.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:49 smithi143 ceph-mon[31461]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:51.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:51 smithi111 ceph-mon[31001]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:51.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:51 smithi097 ceph-mon[28127]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:51 smithi143 ceph-mon[31461]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:53.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:53 smithi097 ceph-mon[28127]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:53.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:24:53.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:53 smithi111 ceph-mon[31001]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:24:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:53 smithi143 ceph-mon[31461]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:24:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:54.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:54 smithi111 ceph-mon[31001]: Deploying daemon osd.3 on smithi097 2024-04-26T21:24:54.946 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:54 smithi097 ceph-mon[28127]: Deploying daemon osd.3 on smithi097 2024-04-26T21:24:55.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:54 smithi143 ceph-mon[31461]: Deploying daemon osd.3 on smithi097 2024-04-26T21:24:55.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:55 smithi111 ceph-mon[31001]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:55 smithi143 ceph-mon[31461]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:55 smithi097 ceph-mon[28127]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:57 smithi097 ceph-mon[28127]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:57.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:57 smithi111 ceph-mon[31001]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:57 smithi143 ceph-mon[31461]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:24:59.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:24:59 smithi111 ceph-mon[31001]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:00.007 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:24:59 smithi097 ceph-mon[28127]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:24:59 smithi143 ceph-mon[31461]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:01.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:01 smithi097 ceph-mon[28127]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:01 smithi143 ceph-mon[31461]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:01 smithi111 ceph-mon[31001]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:02.737 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:02.737 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:02.737 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:03.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:03 smithi143 ceph-mon[31461]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:04.005 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:03 smithi097 ceph-mon[28127]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:03 smithi111 ceph-mon[31001]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:05.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:05 smithi097 ceph-mon[28127]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:05.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:05.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:05.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:05 smithi097 ceph-mon[28127]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:25:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:05 smithi143 ceph-mon[31461]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:05 smithi143 ceph-mon[31461]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:25:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:05 smithi111 ceph-mon[31001]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:05 smithi111 ceph-mon[31001]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:25:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.564+0000 7fa9ef400700 1 -- 172.21.15.97:0/3187524235 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9f8063860 con 0x7fa9dc0a3bb0 2024-04-26T21:25:06.566 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-04-26T21:25:06.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 msgr2=0x7fa9dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8008040 comp rx=0 tx=0).stop 2024-04-26T21:25:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 msgr2=0x7fa9f81a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009c00 tx=0x7fa9f000bd60 comp rx=0 tx=0).stop 2024-04-26T21:25:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 shutdown_connections 2024-04-26T21:25:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f81056c0 0x7fa9f819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9f8107260 0x7fa9f81a2cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 --2- 172.21.15.97:0/3187524235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9f81068c0 0x7fa9f819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 >> 172.21.15.97:0/3187524235 conn(0x7fa9f8100c80 msgr2=0x7fa9f8109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 shutdown_connections 2024-04-26T21:25:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:25:06.568+0000 7fa9fdc00700 1 -- 172.21.15.97:0/3187524235 wait complete. 2024-04-26T21:25:07.396 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:25:07.403 INFO:tasks.cephadm:Deploying osd.4 on smithi111 with /dev/vg_nvme/lv_3... 2024-04-26T21:25:07.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.484 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:07 smithi111 ceph-mon[31001]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:07 smithi143 ceph-mon[31461]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:07.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:25:07.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:25:07.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:25:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:07.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:07 smithi097 ceph-mon[28127]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:25:07.851 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:25:08.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:25:08.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:25:08.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] boot 2024-04-26T21:25:08.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:25:08.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:08 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.367 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] boot 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:25:08.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:08 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: from='osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: osd.3 [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] boot 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:25:08.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:08 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:25:09.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:09 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:25:09.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:09 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:25:09.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:09 smithi097 ceph-mon[28127]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:09.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:09 smithi097 ceph-mon[28127]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:25:09.494 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:09 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:25:09.494 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:09 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:25:09.494 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:09 smithi111 ceph-mon[31001]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:09.495 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:09 smithi111 ceph-mon[31001]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:25:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:09 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:25:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:09 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:25:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:09 smithi143 ceph-mon[31461]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:09 smithi143 ceph-mon[31461]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:25:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:25:10.711 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:25:10.711 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-26T21:25:10.711 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-26T21:25:10.711 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0335676 s, 312 MB/s 2024-04-26T21:25:10.711 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-26T21:25:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:11 smithi111 ceph-mon[31001]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:25:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:11 smithi111 ceph-mon[31001]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:11 smithi111 ceph-mon[31001]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:25:11.488 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-04-26T21:25:11.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:11 smithi097 ceph-mon[28127]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:25:11.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:11 smithi097 ceph-mon[28127]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:11.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:11 smithi097 ceph-mon[28127]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:25:11.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:11 smithi143 ceph-mon[31461]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:25:11.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:11 smithi143 ceph-mon[31461]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:11.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:11 smithi143 ceph-mon[31461]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:25:11.913 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:12.851 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:13.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:13.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:13.832 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi097 2024-04-26T21:25:13.832 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T21:25:13.832 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:13.832 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:13.833 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:13.833 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:13.833 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:13 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 -- 172.21.15.111:0/3365384670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0288107160 msgr2=0x7f028810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 --2- 172.21.15.111:0/3365384670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0288107160 0x7f028810da10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f028000b0d0 tx=0x7f028000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 -- 172.21.15.111:0/3365384670 shutdown_connections 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 --2- 172.21.15.111:0/3365384670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0288107160 0x7f028810da10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 --2- 172.21.15.111:0/3365384670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881055c0 0x7f02881059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 --2- 172.21.15.111:0/3365384670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02881067c0 0x7f0288106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.015+0000 7f028d000700 1 -- 172.21.15.111:0/3365384670 >> 172.21.15.111:0/3365384670 conn(0x7f0288100b60 msgr2=0x7f0288102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:14.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.016+0000 7f028d000700 1 -- 172.21.15.111:0/3365384670 shutdown_connections 2024-04-26T21:25:14.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.016+0000 7f028d000700 1 -- 172.21.15.111:0/3365384670 wait complete. 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.017+0000 7f028d000700 1 Processor -- start 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.017+0000 7f028d000700 1 -- start start 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881067c0 0x7f028819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0288107160 0x7f02881a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0288110f70 con 0x7f02881067c0 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0288110df0 con 0x7f02881055c0 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f028d000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f02881110f0 con 0x7f0288107160 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34814/0 (socket says 172.21.15.111:34814) 2024-04-26T21:25:14.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 -- 172.21.15.111:0/3331811737 learned_addr learned my addr 172.21.15.111:0/3331811737 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0286a00700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881067c0 0x7f028819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287e00700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0288107160 0x7f02881a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 -- 172.21.15.111:0/3331811737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0288107160 msgr2=0x7f02881a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0288107160 0x7f02881a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 -- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881067c0 msgr2=0x7f028819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881067c0 0x7f028819bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.018+0000 7f0287400700 1 -- 172.21.15.111:0/3331811737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02881a3230 con 0x7f02881055c0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f0287400700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f027800d980 tx=0x7f027800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278017070 con 0x7f02881055c0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0278010460 con 0x7f02881055c0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02881a3520 con 0x7f02881055c0 2024-04-26T21:25:14.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027800de60 con 0x7f02881055c0 2024-04-26T21:25:14.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.019+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881a3a10 con 0x7f02881055c0 2024-04-26T21:25:14.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.020+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028806a110 con 0x7f02881055c0 2024-04-26T21:25:14.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.023+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0278009b40 con 0x7f02881055c0 2024-04-26T21:25:14.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.024+0000 7f027fe00700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 0x7f026c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:14.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.024+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f02780cedb0 con 0x7f02881055c0 2024-04-26T21:25:14.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.024+0000 7f0286a00700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 0x7f026c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:14.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.025+0000 7f0286a00700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 0x7f026c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0274005950 tx=0x7f02740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:14.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.027+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f027809c910 con 0x7f02881055c0 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi097 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:13 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:14.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi097 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:14.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:14.246+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0288063860 con 0x7f026c0a3c40 2024-04-26T21:25:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:15.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:15.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:15.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:15.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:15 smithi111 ceph-mon[31001]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:15.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:15 smithi111 ceph-mon[31001]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T21:25:16.008 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:15 smithi143 ceph-mon[31461]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:16.008 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:15 smithi143 ceph-mon[31461]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T21:25:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:15 smithi097 ceph-mon[28127]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:15 smithi097 ceph-mon[28127]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T21:25:17.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:17 smithi111 ceph-mon[31001]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-26T21:25:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:17 smithi143 ceph-mon[31461]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-26T21:25:18.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:17 smithi097 ceph-mon[28127]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-26T21:25:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:18 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/512430064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:18 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:18 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]': finished 2024-04-26T21:25:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:18 smithi097 ceph-mon[28127]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:25:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:18 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:18 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/512430064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:18 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:18 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]': finished 2024-04-26T21:25:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:18 smithi111 ceph-mon[31001]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:25:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:18 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:19.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:18 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/512430064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:19.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:18 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]: dispatch 2024-04-26T21:25:19.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:18 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "672a72b1-edd4-4392-b5ed-1167ae4c4205"}]': finished 2024-04-26T21:25:19.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:18 smithi143 ceph-mon[31461]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:25:19.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:18 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:19 smithi111 ceph-mon[31001]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:25:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:19 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/50890802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:19.943 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:19 smithi097 ceph-mon[28127]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:25:19.944 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:19 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/50890802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:19 smithi143 ceph-mon[31461]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:25:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:19 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/50890802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:21.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:21 smithi111 ceph-mon[31001]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:25:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:21 smithi143 ceph-mon[31461]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:25:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:21 smithi097 ceph-mon[28127]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:25:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:23 smithi111 ceph-mon[31001]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:25:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:23 smithi143 ceph-mon[31461]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:25:24.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:23 smithi097 ceph-mon[28127]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:25:25.713 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:25 smithi111 ceph-mon[31001]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:26.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:25 smithi143 ceph-mon[31461]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:26.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:25 smithi097 ceph-mon[28127]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:27 smithi111 ceph-mon[31001]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:25:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:27 smithi143 ceph-mon[31461]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:25:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:27 smithi097 ceph-mon[28127]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:25:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:28.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:28 smithi111 ceph-mon[31001]: Deploying daemon osd.4 on smithi111 2024-04-26T21:25:29.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:28 smithi143 ceph-mon[31461]: Deploying daemon osd.4 on smithi111 2024-04-26T21:25:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:28 smithi097 ceph-mon[28127]: Deploying daemon osd.4 on smithi111 2024-04-26T21:25:29.954 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:29 smithi111 ceph-mon[31001]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:30.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:29 smithi143 ceph-mon[31461]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:30.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:29 smithi097 ceph-mon[28127]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:31.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:31 smithi111 ceph-mon[31001]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:32.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:31 smithi143 ceph-mon[31461]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:32.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:31 smithi097 ceph-mon[28127]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:33.920 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:33 smithi111 ceph-mon[31001]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:34.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:33 smithi143 ceph-mon[31461]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:33 smithi097 ceph-mon[28127]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:36.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:35 smithi143 ceph-mon[31461]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:35 smithi097 ceph-mon[28127]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:35 smithi111 ceph-mon[31001]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:37.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:37 smithi111 ceph-mon[31001]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:38.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:37 smithi143 ceph-mon[31461]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:37 smithi097 ceph-mon[28127]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:39 smithi143 ceph-mon[31461]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:39 smithi097 ceph-mon[28127]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.085 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:39 smithi111 ceph-mon[31001]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:40.086 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.086 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.211+0000 7f027fe00700 1 -- 172.21.15.111:0/3331811737 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0288063860 con 0x7f026c0a3c40 2024-04-26T21:25:40.212 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 4 on host 'smithi111' 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 msgr2=0x7f026c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 0x7f026c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0274005950 tx=0x7f02740058e0 comp rx=0 tx=0).stop 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 msgr2=0x7f028819b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f027800d980 tx=0x7f027800dd40 comp rx=0 tx=0).stop 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 shutdown_connections 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f026c0a3c40 0x7f026c0a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:40.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02881055c0 0x7f028819b5d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:40.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0288107160 0x7f02881a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:40.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 --2- 172.21.15.111:0/3331811737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02881067c0 0x7f028819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:40.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 >> 172.21.15.111:0/3331811737 conn(0x7f0288100b60 msgr2=0x7f0288109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:40.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 shutdown_connections 2024-04-26T21:25:40.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:25:40.214+0000 7f028d000700 1 -- 172.21.15.111:0/3331811737 wait complete. 2024-04-26T21:25:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:40 smithi111 ceph-mon[31001]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:40 smithi111 ceph-mon[31001]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:40.949 DEBUG:teuthology.orchestra.run.smithi111:osd.4> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:25:40.953 INFO:tasks.cephadm:Deploying osd.5 on smithi143 with /dev/vg_nvme/lv_3... 2024-04-26T21:25:40.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:25:41.032 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:40 smithi143 ceph-mon[31461]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:41.032 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:40 smithi143 ceph-mon[31461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:41.032 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:41.033 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:40 smithi097 ceph-mon[28127]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:40 smithi097 ceph-mon[28127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:25:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:41.389 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:41.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:41 smithi143 ceph-mon[31461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:41 smithi111 ceph-mon[31001]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: from='osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:41 smithi097 ceph-mon[28127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:25:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-mon[31001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:25:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-mon[31001]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:25:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:42.893 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4[42231]: 2024-04-26T21:25:42.720+0000 7f8e54a00700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:25:42.893 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:25:42 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4[42231]: 2024-04-26T21:25:42.769+0000 7f8e49e00700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:25:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:42 smithi143 ceph-mon[31461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:25:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:42 smithi143 ceph-mon[31461]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:25:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:42 smithi097 ceph-mon[28127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:25:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:42 smithi097 ceph-mon[28127]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T21:25:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:43.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:25:43.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:25:43.997 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:43.997 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:43.997 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: from='osd.4 ' entity='osd.4' 2024-04-26T21:25:43.997 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: from='osd.4 ' entity='osd.4' 2024-04-26T21:25:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: from='osd.4 ' entity='osd.4' 2024-04-26T21:25:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343123 s, 306 MB/s 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 2024-04-26T21:25:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-04-26T21:25:44.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:44 smithi143 ceph-mon[31461]: osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] boot 2024-04-26T21:25:44.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:44 smithi143 ceph-mon[31461]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:25:44.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:45.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:44 smithi097 ceph-mon[28127]: osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] boot 2024-04-26T21:25:45.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:44 smithi097 ceph-mon[28127]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:25:45.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:44 smithi111 ceph-mon[31001]: osd.4 [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] boot 2024-04-26T21:25:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:44 smithi111 ceph-mon[31001]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:25:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:25:45.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-04-26T21:25:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:45.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:45.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:45.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:45.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:46.034 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi111 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:46.929 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi111 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:47.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi111 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 -- 172.21.15.143:0/2812177799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 msgr2=0x7f59cc108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2812177799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc108b50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009990 tx=0x7f59b8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 -- 172.21.15.143:0/2812177799 shutdown_connections 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2812177799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc103e30 0x7f59cc106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2812177799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc109090 0x7f59cc10b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2812177799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc108b50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 -- 172.21.15.143:0/2812177799 >> 172.21.15.143:0/2812177799 conn(0x7f59cc0fd840 msgr2=0x7f59cc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 -- 172.21.15.143:0/2812177799 shutdown_connections 2024-04-26T21:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.717+0000 7f59d0a00700 1 -- 172.21.15.143:0/2812177799 wait complete. 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.718+0000 7f59d0a00700 1 Processor -- start 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 -- start start 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 0x7f59cc103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc109090 0x7f59cc102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc10ed30 con 0x7f59cc103e30 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc10ebb0 con 0x7f59cc109090 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59d0a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc10eeb0 con 0x7f59cc106760 2024-04-26T21:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59c9e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59c9e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45042/0 (socket says 172.21.15.143:45042) 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.719+0000 7f59c9e00700 1 -- 172.21.15.143:0/2897991513 learned_addr learned my addr 172.21.15.143:0/2897991513 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59ca800700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 0x7f59cc103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59cb200700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc109090 0x7f59cc102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 -- 172.21.15.143:0/2897991513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc109090 msgr2=0x7f59cc102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc109090 0x7f59cc102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 -- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 msgr2=0x7f59cc103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 0x7f59cc103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 -- 172.21.15.143:0/2897991513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc102bd0 con 0x7f59cc106760 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59ca800700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 0x7f59cc103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c9e00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098e0 tx=0x7f59b8009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.720+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8027070 con 0x7f59cc106760 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.721+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59b80040e0 con 0x7f59cc106760 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.721+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc1a33e0 con 0x7f59cc106760 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.721+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b800f460 con 0x7f59cc106760 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.721+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a3830 con 0x7f59cc106760 2024-04-26T21:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.722+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc06a110 con 0x7f59cc106760 2024-04-26T21:25:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.724+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59b802a070 con 0x7f59cc106760 2024-04-26T21:25:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.725+0000 7f59c3400700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:25:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.726+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f59b80d0bd0 con 0x7f59cc106760 2024-04-26T21:25:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.726+0000 7f59ca800700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:25:47.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.727+0000 7f59ca800700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 0x7f59b00a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005fd0 tx=0x7f59bc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:25:47.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.729+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59b8031020 con 0x7f59cc106760 2024-04-26T21:25:47.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:25:47.965+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f59cc063860 con 0x7f59b00a3c40 2024-04-26T21:25:47.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:47 smithi143 ceph-mon[31461]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:47 smithi097 ceph-mon[28127]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:47 smithi111 ceph-mon[31001]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:25:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:25:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:49 smithi143 ceph-mon[31461]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:49 smithi143 ceph-mon[31461]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:50.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:49 smithi097 ceph-mon[28127]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:50.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:49 smithi097 ceph-mon[28127]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:49 smithi111 ceph-mon[31001]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:25:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:49 smithi111 ceph-mon[31001]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:51 smithi143 ceph-mon[31461]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:52.052 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:51 smithi111 ceph-mon[31001]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:51 smithi097 ceph-mon[28127]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/1134571405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]': finished 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:25:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:52 smithi143 ceph-mon[31461]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/1134571405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]': finished 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:25:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:52 smithi097 ceph-mon[28127]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/1134571405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]: dispatch 2024-04-26T21:25:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52a82b9-6a36-4c0a-8287-f853589258c3"}]': finished 2024-04-26T21:25:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:25:53.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:25:53.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:52 smithi111 ceph-mon[31001]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:54.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:53 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/3614913920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:53 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/3614913920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:53 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/3614913920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:25:55.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:54 smithi143 ceph-mon[31461]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:54 smithi097 ceph-mon[28127]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:54 smithi111 ceph-mon[31001]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:57.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:57 smithi143 ceph-mon[31461]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:57.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:57 smithi111 ceph-mon[31001]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:57 smithi097 ceph-mon[28127]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:59.787 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:25:59 smithi143 ceph-mon[31461]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:25:59.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:25:59 smithi111 ceph-mon[31001]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:25:59 smithi097 ceph-mon[28127]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:01 smithi143 ceph-mon[31461]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:26:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:01 smithi111 ceph-mon[31001]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:26:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:01 smithi097 ceph-mon[28127]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:26:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:02.734 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:02 smithi143 ceph-mon[31461]: Deploying daemon osd.5 on smithi143 2024-04-26T21:26:02.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:02 smithi111 ceph-mon[31001]: Deploying daemon osd.5 on smithi143 2024-04-26T21:26:03.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:02 smithi097 ceph-mon[28127]: Deploying daemon osd.5 on smithi143 2024-04-26T21:26:03.896 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:03 smithi143 ceph-mon[31461]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:03 smithi097 ceph-mon[28127]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:03 smithi111 ceph-mon[31001]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:05.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:05 smithi143 ceph-mon[31461]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:06.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:05 smithi097 ceph-mon[28127]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:05 smithi111 ceph-mon[31001]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:07 smithi143 ceph-mon[31461]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:07 smithi097 ceph-mon[28127]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:07 smithi111 ceph-mon[31001]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:09.770 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:09 smithi143 ceph-mon[31461]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:10.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:09 smithi097 ceph-mon[28127]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:09 smithi111 ceph-mon[31001]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:11.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:11 smithi143 ceph-mon[31461]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:11.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:11.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:11 smithi097 ceph-mon[28127]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:11 smithi111 ceph-mon[31001]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:13.894 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:13 smithi111 ceph-mon[31001]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:13.894 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:13 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:13.980 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:13 smithi143 ceph-mon[31461]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:13.980 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:13 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:13 smithi097 ceph-mon[28127]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:13 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:14.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:14.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.752 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:14.752 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:14 smithi143 ceph-mon[31461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:14.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.879+0000 7f59c3400700 1 -- 172.21.15.143:0/2897991513 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59cc063860 con 0x7f59b00a3c40 2024-04-26T21:26:14.880 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 5 on host 'smithi143' 2024-04-26T21:26:14.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 msgr2=0x7f59b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 0x7f59b00a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005fd0 tx=0x7f59bc005e30 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 msgr2=0x7f59cc101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098e0 tx=0x7f59b8009ee0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 shutdown_connections 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f59b00a3c40 0x7f59b00a6100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59cc109090 0x7f59cc102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59cc103e30 0x7f59cc103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 --2- 172.21.15.143:0/2897991513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59cc106760 0x7f59cc101de0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.883+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 >> 172.21.15.143:0/2897991513 conn(0x7f59cc0fd840 msgr2=0x7f59cc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.884+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 shutdown_connections 2024-04-26T21:26:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:26:14.884+0000 7f59d0a00700 1 -- 172.21.15.143:0/2897991513 wait complete. 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:14 smithi111 ceph-mon[31001]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:15.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:14 smithi097 ceph-mon[28127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:26:15.730 DEBUG:teuthology.orchestra.run.smithi143:osd.5> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:26:15.735 INFO:tasks.cephadm:Deploying osd.6 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-26T21:26:15.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.007 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:26:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:26:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:26:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:16.168 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:26:17.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.046 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5[43391]: 2024-04-26T21:26:16.731+0000 7f3a85600700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:26:17.046 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:26:16 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5[43391]: 2024-04-26T21:26:16.775+0000 7f3a7b400700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:26:17.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T21:26:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:26:17.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:26:17.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:17.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: from='osd.5 ' entity='osd.5' 2024-04-26T21:26:17.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] boot 2024-04-26T21:26:17.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:26:17.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:17 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: from='osd.5 ' entity='osd.5' 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] boot 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:26:17.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:17 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:26:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:26:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:26:18.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:18.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: from='osd.5 ' entity='osd.5' 2024-04-26T21:26:18.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: osd.5 [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] boot 2024-04-26T21:26:18.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:26:18.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:17 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357957 s, 293 MB/s 2024-04-26T21:26:18.991 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T21:26:19.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:19 smithi143 ceph-mon[31461]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:19.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:19 smithi143 ceph-mon[31461]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:26:19.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-26T21:26:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:19 smithi097 ceph-mon[28127]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:19 smithi097 ceph-mon[28127]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:26:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:19 smithi111 ceph-mon[31001]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:19 smithi111 ceph-mon[31001]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:26:20.242 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:26:21.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:20 smithi143 ceph-mon[31461]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:20 smithi097 ceph-mon[28127]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:26:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:26:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:21.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:21.144 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:20 smithi111 ceph-mon[31001]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:26:21.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:21 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi143 2024-04-26T21:26:21.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:21 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi143 to 8544M 2024-04-26T21:26:21.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:21 smithi097 ceph-mon[28127]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:21.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:21 smithi097 ceph-mon[28127]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:26:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:21 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi143 2024-04-26T21:26:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:21 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi143 to 8544M 2024-04-26T21:26:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:21 smithi143 ceph-mon[31461]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:21 smithi143 ceph-mon[31461]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:26:22.089 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:21 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi143 2024-04-26T21:26:22.089 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:21 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi143 to 8544M 2024-04-26T21:26:22.089 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:21 smithi111 ceph-mon[31001]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:22.089 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:21 smithi111 ceph-mon[31001]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.516+0000 7fbf82c00700 1 -- 172.21.15.97:0/1495995753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c077450 msgr2=0x7fbf7c0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.516+0000 7fbf82c00700 1 --2- 172.21.15.97:0/1495995753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c077450 0x7fbf7c0758b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009990 tx=0x7fbf6c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 -- 172.21.15.97:0/1495995753 shutdown_connections 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 --2- 172.21.15.97:0/1495995753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c10d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 --2- 172.21.15.97:0/1495995753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c077450 0x7fbf7c0758b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 --2- 172.21.15.97:0/1495995753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c075e80 0x7fbf7c076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 -- 172.21.15.97:0/1495995753 >> 172.21.15.97:0/1495995753 conn(0x7fbf7c0fea20 msgr2=0x7fbf7c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 -- 172.21.15.97:0/1495995753 shutdown_connections 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.517+0000 7fbf82c00700 1 -- 172.21.15.97:0/1495995753 wait complete. 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 Processor -- start 2024-04-26T21:26:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 -- start start 2024-04-26T21:26:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 0x7fbf7c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c1a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c110fc0 con 0x7fbf7c075e80 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c110e40 con 0x7fbf7c077450 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.518+0000 7fbf82c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c111140 con 0x7fbf7c10b350 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40322/0 (socket says 172.21.15.97:40322) 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 -- 172.21.15.97:0/2512426464 learned_addr learned my addr 172.21.15.97:0/2512426464 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7b400700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 0x7fbf7c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf80c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c1a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 -- 172.21.15.97:0/2512426464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 msgr2=0x7fbf7c1a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c1a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 -- 172.21.15.97:0/2512426464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 msgr2=0x7fbf7c19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 0x7fbf7c19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7be00700 1 -- 172.21.15.97:0/2512426464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c1a3400 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf7b400700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 0x7fbf7c19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.519+0000 7fbf80c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c1a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf7be00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0057c0 tx=0x7fbf6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c027070 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf6c0040a0 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c1a3690 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c022d00 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.520+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c1a3b40 con 0x7fbf7c075e80 2024-04-26T21:26:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.521+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c06a110 con 0x7fbf7c075e80 2024-04-26T21:26:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.524+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf6c02a070 con 0x7fbf7c075e80 2024-04-26T21:26:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.525+0000 7fbf78c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 0x7fbf600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.525+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7fbf6c0d0cc0 con 0x7fbf7c075e80 2024-04-26T21:26:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.525+0000 7fbf7b400700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 0x7fbf600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.526+0000 7fbf7b400700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 0x7fbf600a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c077180 tx=0x7fbf68009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.528+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbf6c031020 con 0x7fbf7c075e80 2024-04-26T21:26:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:22.767+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbf7c063860 con 0x7fbf600a3cc0 2024-04-26T21:26:23.990 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:23 smithi097 ceph-mon[28127]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:23.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:23 smithi097 ceph-mon[28127]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:23.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:23.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:23.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:23 smithi143 ceph-mon[31461]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:23 smithi143 ceph-mon[31461]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:23 smithi111 ceph-mon[31001]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:23 smithi111 ceph-mon[31001]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:24.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:25.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:25 smithi097 ceph-mon[28127]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:26:26.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:25 smithi143 ceph-mon[31461]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:26:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:25 smithi111 ceph-mon[31001]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:26:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:26 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]: dispatch 2024-04-26T21:26:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:26 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]': finished 2024-04-26T21:26:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:26 smithi143 ceph-mon[31461]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:26:27.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:26 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:27.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:26 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]: dispatch 2024-04-26T21:26:27.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:26 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]': finished 2024-04-26T21:26:27.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:26 smithi097 ceph-mon[28127]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:26:27.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:27.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:26 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]: dispatch 2024-04-26T21:26:27.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:26 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3535816138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3c68893-2e7a-4b3a-bbce-eb87ba907a25"}]': finished 2024-04-26T21:26:27.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:26 smithi111 ceph-mon[31001]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:26:27.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:26 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:27 smithi143 ceph-mon[31461]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:26:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:27 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2510494026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:26:28.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:27 smithi097 ceph-mon[28127]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:26:28.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:27 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2510494026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:26:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:27 smithi111 ceph-mon[31001]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:26:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:27 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2510494026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:26:30.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:29 smithi143 ceph-mon[31461]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:26:30.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:29 smithi097 ceph-mon[28127]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:26:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:29 smithi111 ceph-mon[31001]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:26:32.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:31 smithi143 ceph-mon[31461]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:26:32.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:31 smithi097 ceph-mon[28127]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:26:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:31 smithi111 ceph-mon[31001]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T21:26:33.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:33 smithi097 ceph-mon[28127]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:26:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:33 smithi143 ceph-mon[31461]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:26:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:33 smithi111 ceph-mon[31001]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:26:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:35 smithi143 ceph-mon[31461]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:35 smithi097 ceph-mon[28127]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:36.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:35 smithi111 ceph-mon[31001]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:26:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:26:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:26:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:37.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:37 smithi097 ceph-mon[28127]: Deploying daemon osd.6 on smithi097 2024-04-26T21:26:37.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:37 smithi097 ceph-mon[28127]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:38.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:37 smithi143 ceph-mon[31461]: Deploying daemon osd.6 on smithi097 2024-04-26T21:26:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:37 smithi143 ceph-mon[31461]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:37 smithi111 ceph-mon[31001]: Deploying daemon osd.6 on smithi097 2024-04-26T21:26:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:37 smithi111 ceph-mon[31001]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:39.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:39 smithi097 ceph-mon[28127]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:40.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:39 smithi143 ceph-mon[31461]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:39 smithi111 ceph-mon[31001]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:41.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:40 smithi111 ceph-mon[31001]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:40 smithi097 ceph-mon[28127]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:40 smithi143 ceph-mon[31461]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:43.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:43 smithi097 ceph-mon[28127]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:43 smithi111 ceph-mon[31001]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:44.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:43 smithi143 ceph-mon[31461]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:45.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:45 smithi143 ceph-mon[31461]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:45.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:45 smithi111 ceph-mon[31001]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:45.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:45 smithi097 ceph-mon[28127]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:45.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:45.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:47.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:47 smithi097 ceph-mon[28127]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:47 smithi111 ceph-mon[31001]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:48.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:47 smithi143 ceph-mon[31461]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:49.268 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 6 on host 'smithi097' 2024-04-26T21:26:49.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.266+0000 7fbf78c00700 1 -- 172.21.15.97:0/2512426464 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf7c063860 con 0x7fbf600a3cc0 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 msgr2=0x7fbf600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 0x7fbf600a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c077180 tx=0x7fbf68009670 comp rx=0 tx=0).stop 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 msgr2=0x7fbf7c19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0057c0 tx=0x7fbf6c0057f0 comp rx=0 tx=0).stop 2024-04-26T21:26:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 shutdown_connections 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbf600a3cc0 0x7fbf600a6180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf7c10b350 0x7fbf7c1a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf7c075e80 0x7fbf7c19b7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 --2- 172.21.15.97:0/2512426464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf7c077450 0x7fbf7c19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 >> 172.21.15.97:0/2512426464 conn(0x7fbf7c0fea20 msgr2=0x7fbf7c107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 shutdown_connections 2024-04-26T21:26:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:26:49.270+0000 7fbf82c00700 1 -- 172.21.15.97:0/2512426464 wait complete. 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:49.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:26:49.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:49.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:49 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:49.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:49.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:49 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:26:49.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:49.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:49.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:49 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:50.186 DEBUG:teuthology.orchestra.run.smithi097:osd.6> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:26:50.193 INFO:tasks.cephadm:Deploying osd.7 on smithi111 with /dev/vg_nvme/lv_2... 2024-04-26T21:26:50.194 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:26:50.640 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:26:50.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:50 smithi111 ceph-mon[31001]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:26:50.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:50 smithi111 ceph-mon[31001]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:26:50.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:50 smithi111 ceph-mon[31001]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:26:50.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:50.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:50 smithi097 ceph-mon[28127]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:26:50.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:50 smithi097 ceph-mon[28127]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:26:50.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:50 smithi097 ceph-mon[28127]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:26:50.666 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:50.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:50 smithi143 ceph-mon[31461]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:26:50.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:50 smithi143 ceph-mon[31461]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:26:50.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:50 smithi143 ceph-mon[31461]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:26:50.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.763 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-mon[28127]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:26:51.763 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-mon[28127]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:26:51.763 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.763 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.763 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-mon[28127]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:51.764 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6[56626]: 2024-04-26T21:26:51.426+0000 7f6db3600700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:26:51.764 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:26:51 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6[56626]: 2024-04-26T21:26:51.509+0000 7f6da9400700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:26:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:51 smithi143 ceph-mon[31461]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:26:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:51 smithi143 ceph-mon[31461]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:26:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:51 smithi143 ceph-mon[31461]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:51.804 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:51 smithi111 ceph-mon[31001]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:26:51.804 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:51 smithi111 ceph-mon[31001]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T21:26:51.804 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.804 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:51.805 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:51 smithi111 ceph-mon[31001]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:52.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:52 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:26:52.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:52 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:26:52.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:52.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:52 smithi143 ceph-mon[31461]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' 2024-04-26T21:26:52.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:52 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:26:52.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:52 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:26:52.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:52.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:52 smithi097 ceph-mon[28127]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' 2024-04-26T21:26:52.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:52 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:26:52.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:52 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:26:52.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:52.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:52 smithi111 ceph-mon[31001]: from='osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941]' entity='osd.6' 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.033565 s, 312 MB/s 2024-04-26T21:26:53.321 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-26T21:26:53.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:53 smithi111 ceph-mon[31001]: osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] boot 2024-04-26T21:26:53.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:53 smithi111 ceph-mon[31001]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:26:53.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:53 smithi111 ceph-mon[31001]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:53.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:53 smithi143 ceph-mon[31461]: osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] boot 2024-04-26T21:26:53.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:53 smithi143 ceph-mon[31461]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:26:53.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:53 smithi143 ceph-mon[31461]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:53.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:53 smithi097 ceph-mon[28127]: osd.6 [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] boot 2024-04-26T21:26:53.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:53 smithi097 ceph-mon[28127]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:26:53.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:26:53.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:53 smithi097 ceph-mon[28127]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:26:54.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-04-26T21:26:54.552 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:26:54.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:54 smithi143 ceph-mon[31461]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:26:54.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:54 smithi097 ceph-mon[28127]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:26:54.834 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:54 smithi111 ceph-mon[31001]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T21:26:55.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:55.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:55.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:55.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:55.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:55.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:55.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:55.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:26:56.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:56 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi097 2024-04-26T21:26:56.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:56 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T21:26:56.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:56 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi097 2024-04-26T21:26:56.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:56 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T21:26:56.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:56 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi097 2024-04-26T21:26:56.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:56 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T21:26:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 -- 172.21.15.111:0/3227737133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 msgr2=0x7fc2e0106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/3227737133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e0106c20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0099c0 tx=0x7fc2cc009cd0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 -- 172.21.15.111:0/3227737133 shutdown_connections 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/3227737133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e0107160 0x7fc2e010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/3227737133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e01055c0 0x7fc2e01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/3227737133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e0106c20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.853+0000 7fc2e5c00700 1 -- 172.21.15.111:0/3227737133 >> 172.21.15.111:0/3227737133 conn(0x7fc2e0100b60 msgr2=0x7fc2e0102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.854+0000 7fc2e5c00700 1 -- 172.21.15.111:0/3227737133 shutdown_connections 2024-04-26T21:26:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.854+0000 7fc2e5c00700 1 -- 172.21.15.111:0/3227737133 wait complete. 2024-04-26T21:26:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 Processor -- start 2024-04-26T21:26:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 -- start start 2024-04-26T21:26:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e01055c0 0x7fc2e019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e0107160 0x7fc2e01a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0110f10 con 0x7fc2e01055c0 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0110d90 con 0x7fc2e01067c0 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.855+0000 7fc2e5c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0111090 con 0x7fc2e0107160 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42174/0 (socket says 172.21.15.111:42174) 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 -- 172.21.15.111:0/2307518113 learned_addr learned my addr 172.21.15.111:0/2307518113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2dfe00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e01055c0 0x7fc2e019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 -- 172.21.15.111:0/2307518113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e0107160 msgr2=0x7fc2e01a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e0107160 0x7fc2e01a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 -- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e01055c0 msgr2=0x7fc2e019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e01055c0 0x7fc2e019b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 -- 172.21.15.111:0/2307518113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e01a3250 con 0x7fc2e01067c0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.856+0000 7fc2df400700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc000c00 tx=0x7fc2cc00f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.857+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc027070 con 0x7fc2e01067c0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.857+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2cc00bcf0 con 0x7fc2e01067c0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.857+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc0037f0 con 0x7fc2e01067c0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.857+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e01a34e0 con 0x7fc2e01067c0 2024-04-26T21:26:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.857+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01a3a10 con 0x7fc2e01067c0 2024-04-26T21:26:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.858+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e006a110 con 0x7fc2e01067c0 2024-04-26T21:26:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.861+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2cc005080 con 0x7fc2e01067c0 2024-04-26T21:26:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.864+0000 7fc2dcc00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 0x7fc2c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:26:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.864+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fc2cc0d07f0 con 0x7fc2e01067c0 2024-04-26T21:26:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.864+0000 7fc2dfe00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 0x7fc2c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:26:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.865+0000 7fc2dfe00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 0x7fc2c40a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005950 tx=0x7fc2d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:26:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:56.866+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc2cc013070 con 0x7fc2e01067c0 2024-04-26T21:26:57.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:26:57.103+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc2e0063860 con 0x7fc2c40a3cc0 2024-04-26T21:26:57.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:57 smithi143 ceph-mon[31461]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:57.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:57.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:57.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:57 smithi097 ceph-mon[28127]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:57.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:57.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:57 smithi111 ceph-mon[31001]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:57.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:26:57.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:26:57.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:58 smithi111 ceph-mon[31001]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:58.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:58 smithi143 ceph-mon[31461]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:58.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:58 smithi097 ceph-mon[28127]: from='client.34259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:26:59.752 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:26:59 smithi111 ceph-mon[31001]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:59.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:26:59 smithi143 ceph-mon[31461]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:26:59.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:26:59 smithi097 ceph-mon[28127]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/242462552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:01.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]': finished 2024-04-26T21:27:01.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:27:01.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/242462552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]': finished 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:27:02.036 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/242462552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]: dispatch 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8c32d3-48dc-4e01-8e3b-127389176afd"}]': finished 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:27:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:02 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/2373029336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:02 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/2373029336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:03.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:02 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/2373029336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:03.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:03 smithi143 ceph-mon[31461]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:03 smithi097 ceph-mon[28127]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:03 smithi111 ceph-mon[31001]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:06.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:05 smithi143 ceph-mon[31461]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:06.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:05 smithi097 ceph-mon[28127]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:05 smithi111 ceph-mon[31001]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:07.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:07 smithi111 ceph-mon[31001]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:08.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:07 smithi143 ceph-mon[31461]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:07 smithi097 ceph-mon[28127]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:09 smithi143 ceph-mon[31461]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:10.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:09 smithi097 ceph-mon[28127]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:10.081 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:09 smithi111 ceph-mon[31001]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:11.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:27:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:27:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:11.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:27:11.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:11.773 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:11 smithi111 ceph-mon[31001]: Deploying daemon osd.7 on smithi111 2024-04-26T21:27:11.773 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:11 smithi111 ceph-mon[31001]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:11 smithi143 ceph-mon[31461]: Deploying daemon osd.7 on smithi111 2024-04-26T21:27:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:11 smithi143 ceph-mon[31461]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:12.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:11 smithi097 ceph-mon[28127]: Deploying daemon osd.7 on smithi111 2024-04-26T21:27:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:11 smithi097 ceph-mon[28127]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:13.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:13 smithi111 ceph-mon[31001]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:13 smithi143 ceph-mon[31461]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:14.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:13 smithi097 ceph-mon[28127]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:16.032 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:15 smithi111 ceph-mon[31001]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:16.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:15 smithi143 ceph-mon[31461]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:16.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:15 smithi097 ceph-mon[28127]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:18.029 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:17 smithi111 ceph-mon[31001]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:17 smithi143 ceph-mon[31461]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:18.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:17 smithi097 ceph-mon[28127]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:19.866 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:19 smithi111 ceph-mon[31001]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:19.866 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:19 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:19.866 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:19 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:19.867 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:19 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:19.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:19 smithi097 ceph-mon[28127]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:19.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:19 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:19.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:19 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:19.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:19 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:19 smithi143 ceph-mon[31461]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:19 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:19 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:19 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:21.638 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:21 smithi111 ceph-mon[31001]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:21 smithi097 ceph-mon[28127]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:22.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:21 smithi143 ceph-mon[31461]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:23.364 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 7 on host 'smithi111' 2024-04-26T21:27:23.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.363+0000 7fc2dcc00700 1 -- 172.21.15.111:0/2307518113 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc2e0063860 con 0x7fc2c40a3cc0 2024-04-26T21:27:23.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 msgr2=0x7fc2c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:23.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 0x7fc2c40a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0005950 tx=0x7fc2d00058e0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 msgr2=0x7fc2e019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc000c00 tx=0x7fc2cc00f7c0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 shutdown_connections 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc2c40a3cc0 0x7fc2c40a6180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e0107160 0x7fc2e01a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e01055c0 0x7fc2e019b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 --2- 172.21.15.111:0/2307518113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2e01067c0 0x7fc2e019bb30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 >> 172.21.15.111:0/2307518113 conn(0x7fc2e0100b60 msgr2=0x7fc2e0109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 shutdown_connections 2024-04-26T21:27:23.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:27:23.366+0000 7fc2e5c00700 1 -- 172.21.15.111:0/2307518113 wait complete. 2024-04-26T21:27:23.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:23 smithi111 ceph-mon[31001]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:23 smithi143 ceph-mon[31461]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:27:23.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:23 smithi097 ceph-mon[28127]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:24.099 DEBUG:teuthology.orchestra.run.smithi111:osd.7> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:27:24.104 INFO:tasks.cephadm:Deploying osd.8 on smithi143 with /dev/vg_nvme/lv_2... 2024-04-26T21:27:24.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:24.351 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7[49312]: 2024-04-26T21:27:24.195+0000 7f563fc00700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:27:24.351 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:27:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7[49312]: 2024-04-26T21:27:24.218+0000 7f563aa00700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:27:24.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:24.549 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:25.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:25 smithi143 ceph-mon[31461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:27:25.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:25 smithi143 ceph-mon[31461]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:27:25.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.507 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:25 smithi143 ceph-mon[31461]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:25.540 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:25 smithi111 ceph-mon[31001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:27:25.540 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:25 smithi111 ceph-mon[31001]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:27:25.540 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.540 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.540 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:25 smithi111 ceph-mon[31001]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:25 smithi097 ceph-mon[28127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:27:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:25 smithi097 ceph-mon[28127]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T21:27:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:25 smithi097 ceph-mon[28127]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] boot 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:26 smithi111 ceph-mon[31001]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] boot 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:26.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:26 smithi143 ceph-mon[31461]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: osd.7 [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] boot 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:27:26.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:26 smithi097 ceph-mon[28127]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:27:27.568 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:27:27.568 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:27:27.568 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-04-26T21:27:27.568 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-04-26T21:27:27.568 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0398638 s, 263 MB/s 2024-04-26T21:27:27.569 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-04-26T21:27:27.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:27 smithi143 ceph-mon[31461]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:27.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:27 smithi143 ceph-mon[31461]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:27:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:27 smithi111 ceph-mon[31001]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:27.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:27 smithi111 ceph-mon[31001]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:27:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:27 smithi097 ceph-mon[28127]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:27 smithi097 ceph-mon[28127]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:27:28.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-04-26T21:27:28.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:28 smithi143 ceph-mon[31461]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:27:28.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:28.874 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:27:28.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:28 smithi111 ceph-mon[31001]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:27:28.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:28 smithi097 ceph-mon[28127]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:27:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi111 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:29.894 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:29.895 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:29.895 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi111 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:27:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:30.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi111 2024-04-26T21:27:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:30.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 -- 172.21.15.143:0/315538014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c107040 msgr2=0x7f318c1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 --2- 172.21.15.143:0/315538014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c107040 0x7f318c1074c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f3184009ca0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 -- 172.21.15.143:0/315538014 shutdown_connections 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 --2- 172.21.15.143:0/315538014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c101d00 0x7f318c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 --2- 172.21.15.143:0/315538014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c107040 0x7f318c1074c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 --2- 172.21.15.143:0/315538014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c104670 0x7f318c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.947+0000 7f3190e00700 1 -- 172.21.15.143:0/315538014 >> 172.21.15.143:0/315538014 conn(0x7f318c0fd940 msgr2=0x7f318c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:30.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.948+0000 7f3190e00700 1 -- 172.21.15.143:0/315538014 shutdown_connections 2024-04-26T21:27:30.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.948+0000 7f3190e00700 1 -- 172.21.15.143:0/315538014 wait complete. 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 Processor -- start 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 -- start start 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c101d00 0x7f318c19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c107040 0x7f318c1a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f318c115370 con 0x7f318c101d00 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1151f0 con 0x7f318c107040 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.949+0000 7f3190e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1154f0 con 0x7f318c104670 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60798/0 (socket says 172.21.15.143:60798) 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 -- 172.21.15.143:0/1480969164 learned_addr learned my addr 172.21.15.143:0/1480969164 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:27:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3189e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c101d00 0x7f318c19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f318a800700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c107040 0x7f318c1a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 -- 172.21.15.143:0/1480969164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c107040 msgr2=0x7f318c1a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c107040 0x7f318c1a6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 -- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c101d00 msgr2=0x7f318c19f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c101d00 0x7f318c19f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 -- 172.21.15.143:0/1480969164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c1a75e0 con 0x7f318c104670 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.950+0000 7f3181400700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f317400c9e0 tx=0x7f317400cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.951+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174007ac0 con 0x7f318c104670 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.951+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3174007c60 con 0x7f318c104670 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.951+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174018660 con 0x7f318c104670 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.951+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c1a78d0 con 0x7f318c104670 2024-04-26T21:27:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.951+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c1a7dc0 con 0x7f318c104670 2024-04-26T21:27:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.952+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c06a110 con 0x7f318c104670 2024-04-26T21:27:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.954+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f317400f450 con 0x7f318c104670 2024-04-26T21:27:30.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.956+0000 7f3183400700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 0x7f31700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:27:30.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.956+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5397+0+0 (secure 0 0 0) 0x7f31740cfb00 con 0x7f318c104670 2024-04-26T21:27:30.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.956+0000 7f3189e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 0x7f31700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:27:30.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.957+0000 7f3189e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 0x7f31700a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f317c009d10 tx=0x7f317c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:27:30.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:30.959+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3174014030 con 0x7f318c104670 2024-04-26T21:27:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:31.196+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f318c063860 con 0x7f31700a3c40 2024-04-26T21:27:31.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:31 smithi097 ceph-mon[28127]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:31.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:27:31.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:27:31.938 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:32.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:31 smithi143 ceph-mon[31461]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:27:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:27:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:31 smithi111 ceph-mon[31001]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:27:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:27:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:32 smithi143 ceph-mon[31461]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:27:33.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:32 smithi097 ceph-mon[28127]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:27:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:32 smithi111 ceph-mon[31001]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:27:33.761 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:33 smithi143 ceph-mon[31461]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:34.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:33 smithi097 ceph-mon[28127]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:33 smithi111 ceph-mon[31001]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:36.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/2312521314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]': finished 2024-04-26T21:27:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:27:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/2312521314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]': finished 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:27:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/2312521314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]: dispatch 2024-04-26T21:27:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bd31888-d587-4094-a159-3c0f46dd71d2"}]': finished 2024-04-26T21:27:36.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:27:36.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:36 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/1212980989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/1212980989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:36 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/1212980989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:27:38.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:37 smithi143 ceph-mon[31461]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:38.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:37 smithi097 ceph-mon[28127]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:37 smithi111 ceph-mon[31001]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:40.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:39 smithi143 ceph-mon[31461]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:40.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:39 smithi097 ceph-mon[28127]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:39 smithi111 ceph-mon[31001]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:41.900 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:41 smithi143 ceph-mon[31461]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:41.947 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:41 smithi111 ceph-mon[31001]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:41 smithi097 ceph-mon[28127]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:43 smithi111 ceph-mon[31001]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:44.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:43 smithi143 ceph-mon[31461]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:44.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:43 smithi097 ceph-mon[28127]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:45.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:45 smithi143 ceph-mon[31461]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:45.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:27:45.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:45.967 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:45 smithi143 ceph-mon[31461]: Deploying daemon osd.8 on smithi143 2024-04-26T21:27:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:45 smithi097 ceph-mon[28127]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:27:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:45 smithi097 ceph-mon[28127]: Deploying daemon osd.8 on smithi143 2024-04-26T21:27:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:45 smithi111 ceph-mon[31001]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:27:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:45 smithi111 ceph-mon[31001]: Deploying daemon osd.8 on smithi143 2024-04-26T21:27:47.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:47 smithi143 ceph-mon[31461]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:48.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:47 smithi097 ceph-mon[28127]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:47 smithi111 ceph-mon[31001]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:49 smithi143 ceph-mon[31461]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:50.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:49 smithi097 ceph-mon[28127]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:49 smithi111 ceph-mon[31001]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:51 smithi143 ceph-mon[31461]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:52.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:51 smithi097 ceph-mon[28127]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:51 smithi111 ceph-mon[31001]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:53 smithi111 ceph-mon[31001]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:53.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:53 smithi143 ceph-mon[31461]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:53 smithi097 ceph-mon[28127]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:55.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:55 smithi143 ceph-mon[31461]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:55 smithi097 ceph-mon[28127]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:55 smithi111 ceph-mon[31001]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:57 smithi143 ceph-mon[31461]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:57 smithi143 ceph-mon[31461]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:57 smithi143 ceph-mon[31461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:57 smithi097 ceph-mon[28127]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:57 smithi097 ceph-mon[28127]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:57 smithi097 ceph-mon[28127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:57 smithi111 ceph-mon[31001]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:57 smithi111 ceph-mon[31001]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:57 smithi111 ceph-mon[31001]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:27:58.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.375+0000 7f3183400700 1 -- 172.21.15.143:0/1480969164 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f318c063860 con 0x7f31700a3c40 2024-04-26T21:27:58.377 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 8 on host 'smithi143' 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 msgr2=0x7f31700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 0x7f31700a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f317c009d10 tx=0x7f317c009040 comp rx=0 tx=0).stop 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 msgr2=0x7f318c19fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f317400c9e0 tx=0x7f317400cda0 comp rx=0 tx=0).stop 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 shutdown_connections 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f31700a3c40 0x7f31700a6100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f318c107040 0x7f318c1a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f318c101d00 0x7f318c19f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 --2- 172.21.15.143:0/1480969164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318c104670 0x7f318c19fec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:27:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 >> 172.21.15.143:0/1480969164 conn(0x7f318c0fd940 msgr2=0x7f318c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:27:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 shutdown_connections 2024-04-26T21:27:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:27:58.379+0000 7f3190e00700 1 -- 172.21.15.143:0/1480969164 wait complete. 2024-04-26T21:27:59.221 DEBUG:teuthology.orchestra.run.smithi143:osd.8> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:27:59.226 INFO:tasks.cephadm:Deploying osd.9 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-26T21:27:59.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:27:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:27:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:27:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:27:59 smithi111 ceph-mon[31001]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:59.500 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:27:59.500 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:27:59.500 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:59.500 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.500 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.501 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.501 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.501 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:27:59 smithi143 ceph-mon[31461]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:59.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:27:59.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:27:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:27:59 smithi097 ceph-mon[28127]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:27:59.673 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:28:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:00 smithi111 ceph-mon[31001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:28:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:00 smithi111 ceph-mon[31001]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:28:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:00 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:00 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:00.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-mon[31461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:28:00.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-mon[31461]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:28:00.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:00.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:00.545 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:28:00.104+0000 7f3b75400700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:28:00.545 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:28:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:28:00.152+0000 7f3b6b200700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:28:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:00 smithi097 ceph-mon[28127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:28:00.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:00 smithi097 ceph-mon[28127]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:28:00.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:00 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:00.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:00 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.245 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: from='osd.8 ' entity='osd.8' 2024-04-26T21:28:01.246 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:01 smithi143 ceph-mon[31461]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: from='osd.8 ' entity='osd.8' 2024-04-26T21:28:01.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:01 smithi097 ceph-mon[28127]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: osdmap e61: 9 total, 8 up, 9 in 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: from='osd.8 ' entity='osd.8' 2024-04-26T21:28:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:01 smithi111 ceph-mon[31001]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:28:02.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:02 smithi111 ceph-mon[31001]: osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] boot 2024-04-26T21:28:02.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:02 smithi111 ceph-mon[31001]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:28:02.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:02 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:02 smithi143 ceph-mon[31461]: osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] boot 2024-04-26T21:28:02.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:02 smithi143 ceph-mon[31461]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:28:02.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:02 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:02 smithi097 ceph-mon[28127]: osd.8 [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] boot 2024-04-26T21:28:02.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:02 smithi097 ceph-mon[28127]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:28:02.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:02 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0356998 s, 294 MB/s 2024-04-26T21:28:02.616 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T21:28:03.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:03 smithi097 ceph-mon[28127]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:28:03.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:03 smithi097 ceph-mon[28127]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:03.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-26T21:28:03.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:03 smithi143 ceph-mon[31461]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:28:03.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:03 smithi143 ceph-mon[31461]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:03.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:03 smithi111 ceph-mon[31001]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:28:03.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:03 smithi111 ceph-mon[31001]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:03.922 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:04.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:04.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:04 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:04.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:04.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:04 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:28:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:04.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:04.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:04.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:04.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:04 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi143 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi143 to 5696M 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:05.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:05 smithi097 ceph-mon[28127]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi143 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi143 to 5696M 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:05.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:05 smithi143 ceph-mon[31461]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:28:05.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi143 2024-04-26T21:28:05.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi143 to 5696M 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:05.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:05 smithi111 ceph-mon[31001]: osdmap e66: 9 total, 9 up, 9 in 2024-04-26T21:28:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 -- 172.21.15.97:0/1270096889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c107160 msgr2=0x7f388c10da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 --2- 172.21.15.97:0/1270096889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c107160 0x7f388c10da10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f388400b0d0 tx=0x7f388400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 -- 172.21.15.97:0/1270096889 shutdown_connections 2024-04-26T21:28:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 --2- 172.21.15.97:0/1270096889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c1055c0 0x7f388c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 --2- 172.21.15.97:0/1270096889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c107160 0x7f388c10da10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 --2- 172.21.15.97:0/1270096889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1067c0 0x7f388c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 -- 172.21.15.97:0/1270096889 >> 172.21.15.97:0/1270096889 conn(0x7f388c100b40 msgr2=0x7f388c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:28:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 -- 172.21.15.97:0/1270096889 shutdown_connections 2024-04-26T21:28:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.914+0000 7f3892e00700 1 -- 172.21.15.97:0/1270096889 wait complete. 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.915+0000 7f3892e00700 1 Processor -- start 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.915+0000 7f3892e00700 1 -- start start 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1055c0 0x7f388c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c107160 0x7f388c1a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f388c110f30 con 0x7f388c1067c0 2024-04-26T21:28:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f388c110db0 con 0x7f388c107160 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1110b0 con 0x7f388c1055c0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43846/0 (socket says 172.21.15.97:43846) 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 -- 172.21.15.97:0/1292177672 learned_addr learned my addr 172.21.15.97:0/1292177672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891a00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1055c0 0x7f388c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3892400700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c107160 0x7f388c1a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 -- 172.21.15.97:0/1292177672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1055c0 msgr2=0x7f388c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1055c0 0x7f388c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 -- 172.21.15.97:0/1292177672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c107160 msgr2=0x7f388c1a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c107160 0x7f388c1a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.916+0000 7f3891000700 1 -- 172.21.15.97:0/1292177672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c1a3180 con 0x7f388c1067c0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f3891000700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f387800c950 tx=0x7f387800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878007ac0 con 0x7f388c1067c0 2024-04-26T21:28:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3878004cc0 con 0x7f388c1067c0 2024-04-26T21:28:05.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a3470 con 0x7f388c1067c0 2024-04-26T21:28:05.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878005570 con 0x7f388c1067c0 2024-04-26T21:28:05.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.917+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a39b0 con 0x7f388c1067c0 2024-04-26T21:28:05.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.918+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c06a110 con 0x7f388c1067c0 2024-04-26T21:28:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.920+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3878007cf0 con 0x7f388c1067c0 2024-04-26T21:28:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.922+0000 7f388a000700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 0x7f38740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.922+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5829+0+0 (secure 0 0 0) 0x7f38780cf740 con 0x7f388c1067c0 2024-04-26T21:28:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.922+0000 7f3891a00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 0x7f38740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.923+0000 7f3891a00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 0x7f38740a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3880005950 tx=0x7f38800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:28:05.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:05.926+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3878014030 con 0x7f388c1067c0 2024-04-26T21:28:06.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:06.162+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f388c063860 con 0x7f38740a3c40 2024-04-26T21:28:06.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:06.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:06.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:06 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:06.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:06.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:06.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:06 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:06.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:06.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:06.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:06 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:07.323 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:07 smithi097 ceph-mon[28127]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:07.323 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:07 smithi097 ceph-mon[28127]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T21:28:07.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:07 smithi111 ceph-mon[31001]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:07.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:07 smithi111 ceph-mon[31001]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T21:28:07.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:07 smithi143 ceph-mon[31461]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:07.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:07 smithi143 ceph-mon[31461]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T21:28:10.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:09 smithi143 ceph-mon[31461]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:28:10.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:09 smithi097 ceph-mon[28127]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:28:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:09 smithi111 ceph-mon[31001]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:10 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]: dispatch 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:10 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]': finished 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:10 smithi111 ceph-mon[31001]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:10 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]: dispatch 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:10 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]': finished 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:10 smithi143 ceph-mon[31461]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:28:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:11.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:10 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]: dispatch 2024-04-26T21:28:11.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:10 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2931548899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1702b9-db9d-48f5-835f-4ababb3bc2ed"}]': finished 2024-04-26T21:28:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:10 smithi097 ceph-mon[28127]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:28:11.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:11.736 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:11 smithi111 ceph-mon[31001]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:28:11.737 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:11 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/803790249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:11 smithi143 ceph-mon[31461]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:28:12.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:11 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/803790249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:11 smithi097 ceph-mon[28127]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T21:28:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:11 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/803790249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:13 smithi143 ceph-mon[31461]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:28:14.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:13 smithi097 ceph-mon[28127]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:28:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:13 smithi111 ceph-mon[31001]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:28:16.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:15 smithi143 ceph-mon[31461]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:28:16.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:15 smithi097 ceph-mon[28127]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:28:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:15 smithi111 ceph-mon[31001]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:28:17.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:17 smithi097 ceph-mon[28127]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:18.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:17 smithi143 ceph-mon[31461]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:17 smithi111 ceph-mon[31001]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:19 smithi143 ceph-mon[31461]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:19 smithi097 ceph-mon[28127]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:19 smithi111 ceph-mon[31001]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:20.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:28:20.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:21.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:28:21.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:28:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:22.020 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:21 smithi097 ceph-mon[28127]: Deploying daemon osd.9 on smithi097 2024-04-26T21:28:22.020 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:21 smithi097 ceph-mon[28127]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:21 smithi143 ceph-mon[31461]: Deploying daemon osd.9 on smithi097 2024-04-26T21:28:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:21 smithi143 ceph-mon[31461]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:22.090 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:21 smithi111 ceph-mon[31001]: Deploying daemon osd.9 on smithi097 2024-04-26T21:28:22.090 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:21 smithi111 ceph-mon[31001]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:23 smithi097 ceph-mon[28127]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:24.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:23 smithi143 ceph-mon[31461]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:23 smithi111 ceph-mon[31001]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:25 smithi143 ceph-mon[31461]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:25 smithi097 ceph-mon[28127]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:25 smithi111 ceph-mon[31001]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:28.010 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:27 smithi097 ceph-mon[28127]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:27 smithi143 ceph-mon[31461]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:27 smithi111 ceph-mon[31001]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:29.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:29.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:29 smithi097 ceph-mon[28127]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:29 smithi143 ceph-mon[31461]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:29 smithi111 ceph-mon[31001]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:32.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:31 smithi143 ceph-mon[31461]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:31 smithi097 ceph-mon[28127]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:31 smithi111 ceph-mon[31001]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:33.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.203+0000 7f388a000700 1 -- 172.21.15.97:0/1292177672 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f388c063860 con 0x7f38740a3c40 2024-04-26T21:28:33.204 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 9 on host 'smithi097' 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.206+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 msgr2=0x7f38740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.206+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 0x7f38740a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3880005950 tx=0x7f38800058e0 comp rx=0 tx=0).stop 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.206+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 msgr2=0x7f388c19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.206+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f387800c950 tx=0x7f387800cd10 comp rx=0 tx=0).stop 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 shutdown_connections 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38740a3c40 0x7f38740a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388c107160 0x7f388c1a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:33.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f388c1055c0 0x7f388c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:33.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 --2- 172.21.15.97:0/1292177672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c1067c0 0x7f388c19ba60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:33.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 >> 172.21.15.97:0/1292177672 conn(0x7f388c100b40 msgr2=0x7f388c109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:28:33.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 shutdown_connections 2024-04-26T21:28:33.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:28:33.207+0000 7f3892e00700 1 -- 172.21.15.97:0/1292177672 wait complete. 2024-04-26T21:28:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:33 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:33 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:33 smithi143 ceph-mon[31461]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:28:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:33 smithi143 ceph-mon[31461]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:33.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:33 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:33 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:33 smithi097 ceph-mon[28127]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:28:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:33 smithi097 ceph-mon[28127]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:33.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:33 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:33 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:33.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:33 smithi111 ceph-mon[31001]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:28:33.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:33 smithi111 ceph-mon[31001]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:34.034 DEBUG:teuthology.orchestra.run.smithi097:osd.9> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:28:34.038 INFO:tasks.cephadm:Deploying osd.10 on smithi111 with /dev/vg_nvme/lv_1... 2024-04-26T21:28:34.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:28:34.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:28:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:28:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:28:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:34.314 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:28:34.186+0000 7f363e800700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T21:28:34.314 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:28:34 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:28:34.205+0000 7f3639600700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:34.468 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:28:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:28:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:28:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T21:28:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:35 smithi097 ceph-mon[28127]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:28:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:35 smithi097 ceph-mon[28127]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:28:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:35 smithi097 ceph-mon[28127]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:35 smithi111 ceph-mon[31001]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:28:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:35 smithi111 ceph-mon[31001]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:28:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:35 smithi111 ceph-mon[31001]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:35.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:35 smithi143 ceph-mon[31461]: from='osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T21:28:35.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:35 smithi143 ceph-mon[31461]: osdmap e69: 10 total, 9 up, 10 in 2024-04-26T21:28:35.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:35.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:35 smithi143 ceph-mon[31461]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:28:36.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:36 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:28:36.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:36 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:28:36.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:36 smithi097 ceph-mon[28127]: osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] boot 2024-04-26T21:28:36.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:36 smithi097 ceph-mon[28127]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:28:36.315 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:36.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:36 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:28:36.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:36 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:28:36.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:36 smithi111 ceph-mon[31001]: osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] boot 2024-04-26T21:28:36.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:36 smithi111 ceph-mon[31001]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:28:36.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:36 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:28:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:36 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:28:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:36 smithi143 ceph-mon[31461]: osd.9 [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] boot 2024-04-26T21:28:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:36 smithi143 ceph-mon[31461]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:28:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:28:37.411 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:28:37.412 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:28:37.412 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-26T21:28:37.412 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-26T21:28:37.412 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0342622 s, 306 MB/s 2024-04-26T21:28:37.412 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-26T21:28:37.458 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:37 smithi111 ceph-mon[31001]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:28:37.458 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:37 smithi111 ceph-mon[31001]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:37.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:37 smithi143 ceph-mon[31461]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:28:37.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:37 smithi143 ceph-mon[31461]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:37.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:37 smithi097 ceph-mon[28127]: osdmap e71: 10 total, 10 up, 10 in 2024-04-26T21:28:37.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:37 smithi097 ceph-mon[28127]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:38.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-04-26T21:28:38.628 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:28:39.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi097 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.807 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi097 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:39.808 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:39.809 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.809 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.809 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi097 2024-04-26T21:28:39.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:39.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 -- 172.21.15.111:0/764727064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18101d00 msgr2=0x7fbd18102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/764727064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18101d00 0x7fbd18102120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009990 tx=0x7fbd0c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 -- 172.21.15.111:0/764727064 shutdown_connections 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/764727064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18101d00 0x7fbd18102120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/764727064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18103020 0x7fbd181034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/764727064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18102660 0x7fbd18102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.824+0000 7fbd1fc00700 1 -- 172.21.15.111:0/764727064 >> 172.21.15.111:0/764727064 conn(0x7fbd180fd940 msgr2=0x7fbd180ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:28:40.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.825+0000 7fbd1fc00700 1 -- 172.21.15.111:0/764727064 shutdown_connections 2024-04-26T21:28:40.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.825+0000 7fbd1fc00700 1 -- 172.21.15.111:0/764727064 wait complete. 2024-04-26T21:28:40.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 Processor -- start 2024-04-26T21:28:40.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 -- start start 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 0x7fbd1819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18102660 0x7fbd1819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18110fc0 con 0x7fbd18101d00 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18110e40 con 0x7fbd18103020 2024-04-26T21:28:40.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.826+0000 7fbd1fc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18111140 con 0x7fbd18102660 2024-04-26T21:28:40.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:40.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51970/0 (socket says 172.21.15.111:51970) 2024-04-26T21:28:40.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 -- 172.21.15.111:0/635106568 learned_addr learned my addr 172.21.15.111:0/635106568 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-26T21:28:40.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1d200700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 0x7fbd1819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:40.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd17e00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18102660 0x7fbd1819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 -- 172.21.15.111:0/635106568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18102660 msgr2=0x7fbd1819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18102660 0x7fbd1819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.827+0000 7fbd1dc00700 1 -- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 msgr2=0x7fbd1819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1dc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 0x7fbd1819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1dc00700 1 -- 172.21.15.111:0/635106568 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd181a3300 con 0x7fbd18103020 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1d200700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 0x7fbd1819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1dc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd10007ee0 tx=0x7fbd1000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000dbd0 con 0x7fbd18103020 2024-04-26T21:28:40.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd1000dd70 con 0x7fbd18103020 2024-04-26T21:28:40.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100058d0 con 0x7fbd18103020 2024-04-26T21:28:40.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181a35f0 con 0x7fbd18103020 2024-04-26T21:28:40.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.828+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a3ae0 con 0x7fbd18103020 2024-04-26T21:28:40.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.829+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1806a110 con 0x7fbd18103020 2024-04-26T21:28:40.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.832+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd1001a020 con 0x7fbd18103020 2024-04-26T21:28:40.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.833+0000 7fbd15600700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:28:40.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.834+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7fbd100cf4b0 con 0x7fbd18103020 2024-04-26T21:28:40.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.834+0000 7fbd1d200700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:28:40.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.835+0000 7fbd1d200700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b5c0 tx=0x7fbd0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:28:40.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:40.838+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbd10010070 con 0x7fbd18103020 2024-04-26T21:28:41.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:28:41.076+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbd18063860 con 0x7fbd000a3c40 2024-04-26T21:28:41.756 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:41 smithi111 ceph-mon[31001]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:41.756 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:41.757 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:41.757 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:41 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:41 smithi143 ceph-mon[31461]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:41 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:41 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:41 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:41 smithi097 ceph-mon[28127]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:41 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:28:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:41 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:28:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:41 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:43.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:42 smithi143 ceph-mon[31461]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:42 smithi097 ceph-mon[28127]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:43.067 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:42 smithi111 ceph-mon[31001]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:28:43.913 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:43 smithi111 ceph-mon[31001]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:44.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:43 smithi143 ceph-mon[31461]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:44.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:43 smithi097 ceph-mon[28127]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/3313072493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]': finished 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:28:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/3313072493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]': finished 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:28:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/3313072493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]: dispatch 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db65ed02-3eb7-4c51-a807-bf051e380b87"}]': finished 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:28:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:28:47.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:46 smithi143 ceph-mon[31461]: from='client.? 172.21.15.111:0/1813026290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:47.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:46 smithi097 ceph-mon[28127]: from='client.? 172.21.15.111:0/1813026290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:47.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:46 smithi111 ceph-mon[31001]: from='client.? 172.21.15.111:0/1813026290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:28:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:47 smithi143 ceph-mon[31461]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:47 smithi097 ceph-mon[28127]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:47 smithi111 ceph-mon[31001]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:50.008 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:49 smithi111 ceph-mon[31001]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:49 smithi143 ceph-mon[31461]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:50.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:49 smithi097 ceph-mon[28127]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:51.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:51 smithi097 ceph-mon[28127]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:52.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:51 smithi111 ceph-mon[31001]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:52.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:51 smithi143 ceph-mon[31461]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:53.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:53 smithi111 ceph-mon[31001]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:54.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:53 smithi143 ceph-mon[31461]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:53 smithi097 ceph-mon[28127]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:55 smithi143 ceph-mon[31461]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:28:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:55 smithi097 ceph-mon[28127]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:28:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:55 smithi111 ceph-mon[31001]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:28:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:57.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:56 smithi143 ceph-mon[31461]: Deploying daemon osd.10 on smithi111 2024-04-26T21:28:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:56 smithi097 ceph-mon[28127]: Deploying daemon osd.10 on smithi111 2024-04-26T21:28:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:56 smithi111 ceph-mon[31001]: Deploying daemon osd.10 on smithi111 2024-04-26T21:28:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:57 smithi143 ceph-mon[31461]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:58.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:57 smithi097 ceph-mon[28127]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:28:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:57 smithi111 ceph-mon[31001]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:28:59 smithi143 ceph-mon[31461]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:00.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:28:59 smithi097 ceph-mon[28127]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:28:59 smithi111 ceph-mon[31001]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:01.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:01 smithi111 ceph-mon[31001]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:01 smithi143 ceph-mon[31461]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:02.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:01 smithi097 ceph-mon[28127]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:04.039 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:03 smithi111 ceph-mon[31001]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:04.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:03 smithi143 ceph-mon[31461]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:03 smithi097 ceph-mon[28127]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:05.516 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.516 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.516 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:05 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:05.516 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:05 smithi111 ceph-mon[31001]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:05.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:05 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:05.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:05 smithi097 ceph-mon[28127]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:05.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:05.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:05 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:05.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:05 smithi143 ceph-mon[31461]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:07.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:07 smithi111 ceph-mon[31001]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:07 smithi143 ceph-mon[31461]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:07 smithi097 ceph-mon[28127]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:09.255 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:09 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.255 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:09 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.255 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:09 smithi111 ceph-mon[31001]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.256 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:09 smithi111 ceph-mon[31001]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.256 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:09 smithi111 ceph-mon[31001]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:09.362 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 10 on host 'smithi111' 2024-04-26T21:29:09.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.361+0000 7fbd15600700 1 -- 172.21.15.111:0/635106568 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbd18063860 con 0x7fbd000a3c40 2024-04-26T21:29:09.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 msgr2=0x7fbd000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:09.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b5c0 tx=0x7fbd0c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 msgr2=0x7fbd181a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:09.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd10007ee0 tx=0x7fbd1000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 shutdown_connections 2024-04-26T21:29:09.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18103020 0x7fbd181a2c60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd18101d00 0x7fbd1819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 --2- 172.21.15.111:0/635106568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd18102660 0x7fbd1819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 >> 172.21.15.111:0/635106568 conn(0x7fbd180fd940 msgr2=0x7fbd180ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.364+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 shutdown_connections 2024-04-26T21:29:09.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-26T21:29:09.365+0000 7fbd1fc00700 1 -- 172.21.15.111:0/635106568 wait complete. 2024-04-26T21:29:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:09 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:09 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:09 smithi143 ceph-mon[31461]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:09 smithi143 ceph-mon[31461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:09 smithi143 ceph-mon[31461]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:09 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:09 smithi097 ceph-mon[28127]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:09 smithi097 ceph-mon[28127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:29:09.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:09 smithi097 ceph-mon[28127]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:10.102 DEBUG:teuthology.orchestra.run.smithi111:osd.10> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:29:10.107 INFO:tasks.cephadm:Deploying osd.11 on smithi143 with /dev/vg_nvme/lv_1... 2024-04-26T21:29:10.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:10.329 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:10.330 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:29:10.177+0000 7f32afe00700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T21:29:10.330 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:29:10 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:29:10.198+0000 7f32aac00700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:29:10.532 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:29:10.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:10.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:10 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:10.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:10 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:11.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:29:11.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:29:11.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: pgmap v299: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] boot 2024-04-26T21:29:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:29:11.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: pgmap v299: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] boot 2024-04-26T21:29:11.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:29:11.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: osdmap e74: 11 total, 10 up, 11 in 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: pgmap v299: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:29:11.592 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: osd.10 [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] boot 2024-04-26T21:29:11.593 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:29:11.593 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:29:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:12 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:29:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:12 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:29:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:12 smithi143 ceph-mon[31461]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:29:12.420 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:12 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:29:12.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:12 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:29:12.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:12 smithi111 ceph-mon[31001]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:29:12.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:12 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:29:12.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:12 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:29:12.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:12 smithi097 ceph-mon[28127]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0328518 s, 319 MB/s 2024-04-26T21:29:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-04-26T21:29:13.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:13 smithi143 ceph-mon[31461]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:13.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:13 smithi143 ceph-mon[31461]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:29:13.946 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:13 smithi111 ceph-mon[31001]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:13.946 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:13 smithi111 ceph-mon[31001]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:29:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:13 smithi097 ceph-mon[28127]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:13 smithi097 ceph-mon[28127]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:29:14.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-04-26T21:29:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi111 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.401 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:15.402 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:29:15.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:15.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi111 2024-04-26T21:29:15.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:15.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:29:15.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: pgmap v305: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:15.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi111 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:15.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:16.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:16 smithi143 ceph-mon[31461]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:29:16.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:16.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:16.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:16.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:16.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:16 smithi097 ceph-mon[28127]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:29:16.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:16.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:16.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:16.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:16 smithi111 ceph-mon[31001]: osdmap e79: 11 total, 11 up, 11 in 2024-04-26T21:29:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 -- 172.21.15.143:0/4240334175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f561410b350 msgr2=0x7f561410d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 --2- 172.21.15.143:0/4240334175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f561410b350 0x7f561410d7e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f560c00b0a0 tx=0x7f560c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 -- 172.21.15.143:0/4240334175 shutdown_connections 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 --2- 172.21.15.143:0/4240334175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 0x7f56140758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 --2- 172.21.15.143:0/4240334175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f561410b350 0x7f561410d7e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 --2- 172.21.15.143:0/4240334175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5614075e80 0x7f5614076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 -- 172.21.15.143:0/4240334175 >> 172.21.15.143:0/4240334175 conn(0x7f56140fea20 msgr2=0x7f5614100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:17.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 -- 172.21.15.143:0/4240334175 shutdown_connections 2024-04-26T21:29:17.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.060+0000 7f561aa00700 1 -- 172.21.15.143:0/4240334175 wait complete. 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 Processor -- start 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 -- start start 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 0x7f561419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561410b350 0x7f56141a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111030 con 0x7f561410b350 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5614110eb0 con 0x7f5614077450 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.061+0000 7f561aa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f56141111b0 con 0x7f5614075e80 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:17.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33450/0 (socket says 172.21.15.143:33450) 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 -- 172.21.15.143:0/1840440324 learned_addr learned my addr 172.21.15.143:0/1840440324 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613400700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 0x7f561419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5618a00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561410b350 0x7f56141a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 -- 172.21.15.143:0/1840440324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 msgr2=0x7f561419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 0x7f561419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 -- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561410b350 msgr2=0x7f56141a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561410b350 0x7f56141a2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.062+0000 7f5613e00700 1 -- 172.21.15.143:0/1840440324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141a32e0 con 0x7f5614075e80 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f5613e00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f560400c9a0 tx=0x7f560400cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604007ac0 con 0x7f5614075e80 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5604004cc0 con 0x7f5614075e80 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141a35d0 con 0x7f5614075e80 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604005580 con 0x7f5614075e80 2024-04-26T21:29:17.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.063+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a3b10 con 0x7f5614075e80 2024-04-26T21:29:17.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.064+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561406a110 con 0x7f5614075e80 2024-04-26T21:29:17.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.066+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5604007cf0 con 0x7f5614075e80 2024-04-26T21:29:17.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.067+0000 7f5610c00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 0x7f55f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:17.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.068+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6694+0+0 (secure 0 0 0) 0x7f56040cfa30 con 0x7f5614075e80 2024-04-26T21:29:17.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.068+0000 7f5613400700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 0x7f55f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:17.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.069+0000 7f5613400700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 0x7f55f80a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5614077180 tx=0x7f5600005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:17.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.072+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5604014030 con 0x7f5614075e80 2024-04-26T21:29:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:17.313+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5614063860 con 0x7f55f80a3cc0 2024-04-26T21:29:17.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:17 smithi143 ceph-mon[31461]: pgmap v307: 1 pgs: 1 peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-26T21:29:17.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:17 smithi097 ceph-mon[28127]: pgmap v307: 1 pgs: 1 peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-26T21:29:17.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:17 smithi111 ceph-mon[31001]: pgmap v307: 1 pgs: 1 peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-26T21:29:18.478 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:18 smithi143 ceph-mon[31461]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:29:18.479 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:18 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:29:18.479 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:18 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:29:18.479 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:18 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:18 smithi097 ceph-mon[28127]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:29:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:18 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:29:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:18 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:29:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:18 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:18.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:18 smithi111 ceph-mon[31001]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:29:18.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:18 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:29:18.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:18 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:29:18.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:18 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:19.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:19 smithi143 ceph-mon[31461]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:29:19.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:19 smithi097 ceph-mon[28127]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:29:19.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:19 smithi111 ceph-mon[31001]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:29:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/29649761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]': finished 2024-04-26T21:29:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:29:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/29649761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]': finished 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:29:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/29649761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]: dispatch 2024-04-26T21:29:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cd4c20-11c9-426a-9504-09d4fe119a5a"}]': finished 2024-04-26T21:29:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:29:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:23.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:22 smithi143 ceph-mon[31461]: from='client.? 172.21.15.143:0/4065374010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:29:23.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.143:0/4065374010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:29:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:22 smithi111 ceph-mon[31001]: from='client.? 172.21.15.143:0/4065374010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:29:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:23 smithi143 ceph-mon[31461]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:24.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:23 smithi097 ceph-mon[28127]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:23 smithi111 ceph-mon[31001]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:29:26.024 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:25 smithi143 ceph-mon[31461]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:29:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:25 smithi097 ceph-mon[28127]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:29:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:25 smithi111 ceph-mon[31001]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T21:29:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:27 smithi143 ceph-mon[31461]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:27 smithi097 ceph-mon[28127]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:27 smithi111 ceph-mon[31001]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:29 smithi143 ceph-mon[31461]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:30.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:29 smithi097 ceph-mon[28127]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:29 smithi111 ceph-mon[31001]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:31.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:31 smithi097 ceph-mon[28127]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:31.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:29:31.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:31 smithi143 ceph-mon[31461]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:29:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:31 smithi111 ceph-mon[31001]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:29:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:33.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:32 smithi143 ceph-mon[31461]: Deploying daemon osd.11 on smithi143 2024-04-26T21:29:33.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:32 smithi097 ceph-mon[28127]: Deploying daemon osd.11 on smithi143 2024-04-26T21:29:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:32 smithi111 ceph-mon[31001]: Deploying daemon osd.11 on smithi143 2024-04-26T21:29:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:33 smithi143 ceph-mon[31461]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:34.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:33 smithi097 ceph-mon[28127]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:33 smithi111 ceph-mon[31001]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:35 smithi143 ceph-mon[31461]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:35 smithi097 ceph-mon[28127]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:35 smithi111 ceph-mon[31001]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:38.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:37 smithi143 ceph-mon[31461]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:37 smithi097 ceph-mon[28127]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:37 smithi111 ceph-mon[31001]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:39 smithi097 ceph-mon[28127]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:39 smithi111 ceph-mon[31001]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:39 smithi143 ceph-mon[31461]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:41 smithi097 ceph-mon[28127]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:41 smithi111 ceph-mon[31001]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:41 smithi143 ceph-mon[31461]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:44.005 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.006 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:43 smithi143 ceph-mon[31461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:44.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:44.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:44.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:43 smithi097 ceph-mon[28127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:44.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:44.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:44.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:43 smithi111 ceph-mon[31001]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:29:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.421+0000 7f5610c00700 1 -- 172.21.15.143:0/1840440324 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5614063860 con 0x7f55f80a3cc0 2024-04-26T21:29:44.422 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 11 on host 'smithi143' 2024-04-26T21:29:44.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 msgr2=0x7f55f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 0x7f55f80a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5614077180 tx=0x7f5600005d70 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 msgr2=0x7f561419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f560400c9a0 tx=0x7f560400cd60 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 shutdown_connections 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f55f80a3cc0 0x7f55f80a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561410b350 0x7f56141a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5614075e80 0x7f561419b6d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 --2- 172.21.15.143:0/1840440324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5614077450 0x7f561419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 >> 172.21.15.143:0/1840440324 conn(0x7f56140fea20 msgr2=0x7f5614107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 shutdown_connections 2024-04-26T21:29:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:44.425+0000 7f561aa00700 1 -- 172.21.15.143:0/1840440324 wait complete. 2024-04-26T21:29:45.228 DEBUG:teuthology.orchestra.run.smithi143:osd.11> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:29:45.233 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:29:45.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:29:45.460 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.460 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:45.460 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:45 smithi143 ceph-mon[31461]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:45 smithi097 ceph-mon[28127]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:45.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:45.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:45.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:29:45.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:29:45.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:45.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-04-26T21:29:45.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:45 smithi111 ceph-mon[31001]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:45.675 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:29:46.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-mon[31461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:29:46.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-mon[31461]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:29:46.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:46.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:46.671 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:29:46.477+0000 7f7678400700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:29:46.671 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:29:46 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:29:46.548+0000 7f766d800700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:29:46.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:46 smithi097 ceph-mon[28127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:29:46.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:46 smithi097 ceph-mon[28127]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:29:46.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:46.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:46.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:46 smithi111 ceph-mon[31001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-04-26T21:29:46.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:46 smithi111 ceph-mon[31001]: osdmap e82: 12 total, 11 up, 12 in 2024-04-26T21:29:46.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:46.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:47.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:47 smithi097 ceph-mon[28127]: purged_snaps scrub starts 2024-04-26T21:29:47.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:47 smithi097 ceph-mon[28127]: purged_snaps scrub ok 2024-04-26T21:29:47.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:47 smithi097 ceph-mon[28127]: from='osd.11 ' entity='osd.11' 2024-04-26T21:29:47.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:47.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:47 smithi097 ceph-mon[28127]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:47.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 -- 172.21.15.97:0/718800257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1091f0 msgr2=0x7f3edc10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 --2- 172.21.15.97:0/718800257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1091f0 0x7f3edc10b5e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b0a0 tx=0x7f3ed400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 -- 172.21.15.97:0/718800257 shutdown_connections 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 --2- 172.21.15.97:0/718800257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc103f90 0x7f3edc106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 --2- 172.21.15.97:0/718800257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1091f0 0x7f3edc10b5e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 --2- 172.21.15.97:0/718800257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc1068c0 0x7f3edc108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 -- 172.21.15.97:0/718800257 >> 172.21.15.97:0/718800257 conn(0x7f3edc0fd920 msgr2=0x7f3edc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 -- 172.21.15.97:0/718800257 shutdown_connections 2024-04-26T21:29:47.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.721+0000 7f3ee1800700 1 -- 172.21.15.97:0/718800257 wait complete. 2024-04-26T21:29:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.722+0000 7f3ee1800700 1 Processor -- start 2024-04-26T21:29:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.722+0000 7f3ee1800700 1 -- start start 2024-04-26T21:29:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc103f90 0x7f3edc19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 0x7f3edc1a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc10ee00 con 0x7f3edc1068c0 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc10ec80 con 0x7f3edc1091f0 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ee1800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc10ef80 con 0x7f3edc103f90 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ed2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ed2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40366/0 (socket says 172.21.15.97:40366) 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3ed2000700 1 -- 172.21.15.97:0/2064780831 learned_addr learned my addr 172.21.15.97:0/2064780831 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3edaa00700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc103f90 0x7f3edc19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.723+0000 7f3edb400700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 0x7f3edc1a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 -- 172.21.15.97:0/2064780831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc103f90 msgr2=0x7f3edc19b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc103f90 0x7f3edc19b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 -- 172.21.15.97:0/2064780831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 msgr2=0x7f3edc1a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 0x7f3edc1a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 -- 172.21.15.97:0/2064780831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc1a33b0 con 0x7f3edc1068c0 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3edb400700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 0x7f3edc1a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed2000700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d9b0 tx=0x7f3ec400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4017070 con 0x7f3edc1068c0 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc1a36a0 con 0x7f3edc1068c0 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ec4005020 con 0x7f3edc1068c0 2024-04-26T21:29:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.724+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009ab0 con 0x7f3edc1068c0 2024-04-26T21:29:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.725+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc1a3b90 con 0x7f3edc1068c0 2024-04-26T21:29:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.726+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc06a110 con 0x7f3edc1068c0 2024-04-26T21:29:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.728+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ec40051d0 con 0x7f3edc1068c0 2024-04-26T21:29:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.729+0000 7f3ed3e00700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 0x7f3ec00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.729+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f3ec40cfb30 con 0x7f3edc1068c0 2024-04-26T21:29:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.729+0000 7f3edaa00700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 0x7f3ec00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.730+0000 7f3edaa00700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 0x7f3ec00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc005950 tx=0x7f3ecc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.734+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3ec4020030 con 0x7f3edc1068c0 2024-04-26T21:29:47.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:47 smithi143 ceph-mon[31461]: purged_snaps scrub starts 2024-04-26T21:29:47.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:47 smithi143 ceph-mon[31461]: purged_snaps scrub ok 2024-04-26T21:29:47.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:47 smithi143 ceph-mon[31461]: from='osd.11 ' entity='osd.11' 2024-04-26T21:29:47.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:47.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:47 smithi143 ceph-mon[31461]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:47 smithi111 ceph-mon[31001]: purged_snaps scrub starts 2024-04-26T21:29:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:47 smithi111 ceph-mon[31001]: purged_snaps scrub ok 2024-04-26T21:29:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:47 smithi111 ceph-mon[31001]: from='osd.11 ' entity='osd.11' 2024-04-26T21:29:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:47.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:47 smithi111 ceph-mon[31001]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:29:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.965+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3edc02e490 con 0x7f3edc1068c0 2024-04-26T21:29:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.965+0000 7f3ed3e00700 1 -- 172.21.15.97:0/2064780831 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f3ec4096e20 con 0x7f3edc1068c0 2024-04-26T21:29:47.967 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:29:47.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.968+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 msgr2=0x7f3ec00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:47.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.968+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 0x7f3ec00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc005950 tx=0x7f3ecc00b410 comp rx=0 tx=0).stop 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.968+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 msgr2=0x7f3edc19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.968+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d9b0 tx=0x7f3ec400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 shutdown_connections 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3ec00a3c40 0x7f3ec00a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3edc1091f0 0x7f3edc1a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3edc103f90 0x7f3edc19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 --2- 172.21.15.97:0/2064780831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3edc1068c0 0x7f3edc19bc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 >> 172.21.15.97:0/2064780831 conn(0x7f3edc0fd920 msgr2=0x7f3edc103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 shutdown_connections 2024-04-26T21:29:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:47.969+0000 7f3ee1800700 1 -- 172.21.15.97:0/2064780831 wait complete. 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] boot 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2064780831' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:29:48.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:48 smithi097 ceph-mon[28127]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:29:48.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] boot 2024-04-26T21:29:48.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:29:48.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2064780831' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:29:48.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:48 smithi143 ceph-mon[31461]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:29:48.838 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714166987,"num_in_osds":12,"osd_in_since":1714166961,"num_remapped_pgs":0} 2024-04-26T21:29:48.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:29:48.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: osd.11 [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] boot 2024-04-26T21:29:48.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:29:48.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:29:48.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2064780831' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:29:48.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:48 smithi111 ceph-mon[31001]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:29:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:29:49.686 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:49 smithi097 ceph-mon[28127]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:49.771 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:49 smithi143 ceph-mon[31461]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:49.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:49 smithi111 ceph-mon[31001]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:50.958 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: Detected new or changed devices on smithi143 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi143 to 4272M 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:50.959 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:50 smithi097 ceph-mon[28127]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: Detected new or changed devices on smithi143 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi143 to 4272M 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:50 smithi111 ceph-mon[31001]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: Detected new or changed devices on smithi143 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi143 to 4272M 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:29:51.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:50 smithi143 ceph-mon[31461]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:51.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 -- 172.21.15.97:0/621019553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103e30 msgr2=0x7fb5c0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 --2- 172.21.15.97:0/621019553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103e30 0x7fb5c0110290 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b0a0 tx=0x7fb5b800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 -- 172.21.15.97:0/621019553 shutdown_connections 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 --2- 172.21.15.97:0/621019553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c01094c0 0x7fb5c01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 --2- 172.21.15.97:0/621019553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103e30 0x7fb5c0110290 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 --2- 172.21.15.97:0/621019553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c0103470 0x7fb5c01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 -- 172.21.15.97:0/621019553 >> 172.21.15.97:0/621019553 conn(0x7fb5c007a580 msgr2=0x7fb5c007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 -- 172.21.15.97:0/621019553 shutdown_connections 2024-04-26T21:29:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.490+0000 7fb5c6600700 1 -- 172.21.15.97:0/621019553 wait complete. 2024-04-26T21:29:51.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.491+0000 7fb5c6600700 1 Processor -- start 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.491+0000 7fb5c6600700 1 -- start start 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 0x7fb5c01a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c01094c0 0x7fb5c019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01134a0 con 0x7fb5c0103470 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0113320 con 0x7fb5c0103e30 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5c6600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0113620 con 0x7fb5c01094c0 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5bf400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5bf400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46538/0 (socket says 172.21.15.97:46538) 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5bf400700 1 -- 172.21.15.97:0/1981633214 learned_addr learned my addr 172.21.15.97:0/1981633214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5b7e00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 0x7fb5c01a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5bf400700 1 -- 172.21.15.97:0/1981633214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c01094c0 msgr2=0x7fb5c019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.492+0000 7fb5bfe00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c01094c0 0x7fb5c019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bf400700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c01094c0 0x7fb5c019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bf400700 1 -- 172.21.15.97:0/1981633214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 msgr2=0x7fb5c01a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bf400700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 0x7fb5c01a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bf400700 1 -- 172.21.15.97:0/1981633214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c019c180 con 0x7fb5c0103470 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5b7e00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 0x7fb5c01a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:29:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bf400700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0016070 con 0x7fb5c0103470 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5b0010460 con 0x7fb5c0103470 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c019c410 con 0x7fb5c0103470 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b000f620 con 0x7fb5c0103470 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.493+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01a8730 con 0x7fb5c0103470 2024-04-26T21:29:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.495+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c006a110 con 0x7fb5c0103470 2024-04-26T21:29:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.496+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5b0019070 con 0x7fb5c0103470 2024-04-26T21:29:51.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.498+0000 7fb5bcc00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 0x7fb5a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:51.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.498+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb5b00cfb50 con 0x7fb5c0103470 2024-04-26T21:29:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.498+0000 7fb5b7e00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 0x7fb5a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.499+0000 7fb5b7e00700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 0x7fb5a40a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8005fd0 tx=0x7fb5a8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.503+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb5b009c930 con 0x7fb5c0103470 2024-04-26T21:29:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.727+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5c002e490 con 0x7fb5c0103470 2024-04-26T21:29:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.728+0000 7fb5bcc00700 1 -- 172.21.15.97:0/1981633214 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18457 (secure 0 0 0) 0x7fb5b009c230 con 0x7fb5c0103470 2024-04-26T21:29:51.730 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:29:51.730 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:29:48.485798+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.731+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 msgr2=0x7fb5a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.731+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 0x7fb5a40a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8005fd0 tx=0x7fb5a8005ee0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.731+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 msgr2=0x7fb5c01a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.731+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000dd10 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 shutdown_connections 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb5a40a3c40 0x7fb5a40a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c01094c0 0x7fb5c019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0103470 0x7fb5c01a1860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 --2- 172.21.15.97:0/1981633214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c0103e30 0x7fb5c01a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 >> 172.21.15.97:0/1981633214 conn(0x7fb5c007a580 msgr2=0x7fb5c010e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:51.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 shutdown_connections 2024-04-26T21:29:51.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:51.732+0000 7fb5c6600700 1 -- 172.21.15.97:0/1981633214 wait complete. 2024-04-26T21:29:52.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:51 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1981633214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:29:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:51 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1981633214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:29:52.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:51 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1981633214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:29:52.601 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:29:52.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:29:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:52 smithi143 ceph-mon[31461]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:53.045 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:29:53.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:52 smithi097 ceph-mon[28127]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:53.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:52 smithi111 ceph-mon[31001]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 -- 172.21.15.97:0/1764464633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294406ba50 msgr2=0x7f294410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 --2- 172.21.15.97:0/1764464633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294406ba50 0x7f294410f990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f292c009990 tx=0x7f292c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 -- 172.21.15.97:0/1764464633 shutdown_connections 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 --2- 172.21.15.97:0/1764464633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294410fed0 0x7f29441122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 --2- 172.21.15.97:0/1764464633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406b130 0x7f294406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 --2- 172.21.15.97:0/1764464633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294406ba50 0x7f294410f990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 -- 172.21.15.97:0/1764464633 >> 172.21.15.97:0/1764464633 conn(0x7f29440792d0 msgr2=0x7f29440796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 -- 172.21.15.97:0/1764464633 shutdown_connections 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.176+0000 7f294aa00700 1 -- 172.21.15.97:0/1764464633 wait complete. 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.177+0000 7f294aa00700 1 Processor -- start 2024-04-26T21:29:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 -- start start 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294406b130 0x7f29441a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406ba50 0x7f29441a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29441156e0 con 0x7f294410fed0 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2944115560 con 0x7f294406b130 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f294aa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2944115860 con 0x7f294406ba50 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f2948a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f2948a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46560/0 (socket says 172.21.15.97:46560) 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.178+0000 7f2948a00700 1 -- 172.21.15.97:0/2546112939 learned_addr learned my addr 172.21.15.97:0/2546112939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f293be00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406ba50 0x7f29441a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2943e00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294406b130 0x7f29441a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 -- 172.21.15.97:0/2546112939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406ba50 msgr2=0x7f29441a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406ba50 0x7f29441a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 -- 172.21.15.97:0/2546112939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294406b130 msgr2=0x7f29441a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294406b130 0x7f29441a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 -- 172.21.15.97:0/2546112939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294419c200 con 0x7f294410fed0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2948a00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f293c0094f0 tx=0x7f293c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.179+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c004db0 con 0x7f294410fed0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.180+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f293c005020 con 0x7f294410fed0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.180+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c015660 con 0x7f294410fed0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.180+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294419c4f0 con 0x7f294410fed0 2024-04-26T21:29:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.180+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441a87b0 con 0x7f294410fed0 2024-04-26T21:29:55.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.181+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294406a110 con 0x7f294410fed0 2024-04-26T21:29:55.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.183+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f293c0051d0 con 0x7f294410fed0 2024-04-26T21:29:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.184+0000 7f2941600700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 0x7f29280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.184+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f293c0d0290 con 0x7f294410fed0 2024-04-26T21:29:55.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.184+0000 7f2943e00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 0x7f29280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:55.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.185+0000 7f2943e00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 0x7f29280a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2934005950 tx=0x7f29340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:55.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.187+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f293c010070 con 0x7f294410fed0 2024-04-26T21:29:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.413+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f294402e490 con 0x7f294410fed0 2024-04-26T21:29:55.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.414+0000 7f2941600700 1 -- 172.21.15.97:0/2546112939 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f293c09d070 con 0x7f294410fed0 2024-04-26T21:29:55.415 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 msgr2=0x7f29280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 0x7f29280a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2934005950 tx=0x7f29340058e0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 msgr2=0x7f294419baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f293c0094f0 tx=0x7f293c007f90 comp rx=0 tx=0).stop 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 shutdown_connections 2024-04-26T21:29:55.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f29280a3c40 0x7f29280a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f294406b130 0x7f29441a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f294410fed0 0x7f294419baa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 --2- 172.21.15.97:0/2546112939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f294406ba50 0x7f29441a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 >> 172.21.15.97:0/2546112939 conn(0x7f29440792d0 msgr2=0x7f29441114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 shutdown_connections 2024-04-26T21:29:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:29:55.417+0000 7f294aa00700 1 -- 172.21.15.97:0/2546112939 wait complete. 2024-04-26T21:29:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:55 smithi143 ceph-mon[31461]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:55 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2546112939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:29:56.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:55 smithi097 ceph-mon[28127]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:55 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2546112939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:29:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:55 smithi111 ceph-mon[31001]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:55 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2546112939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:29:56.341 INFO:tasks.cephadm:Adding mds.a on smithi097 2024-04-26T21:29:56.341 INFO:tasks.cephadm:Adding mds.d on smithi097 2024-04-26T21:29:56.341 INFO:tasks.cephadm:Adding mds.g on smithi097 2024-04-26T21:29:56.341 INFO:tasks.cephadm:Adding mds.j on smithi097 2024-04-26T21:29:56.341 INFO:tasks.cephadm:Adding mds.b on smithi111 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.e on smithi111 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.h on smithi111 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.k on smithi111 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.c on smithi143 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.f on smithi143 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.i on smithi143 2024-04-26T21:29:56.342 INFO:tasks.cephadm:Adding mds.l on smithi143 2024-04-26T21:29:56.342 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l' 2024-04-26T21:29:56.791 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:29:57.972 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:57 smithi143 ceph-mon[31461]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:58.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:57 smithi097 ceph-mon[28127]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:57 smithi111 ceph-mon[31001]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.986+0000 7f9d83200700 1 -- 172.21.15.143:0/2378637638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1056d0 msgr2=0x7f9d7c105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.986+0000 7f9d83200700 1 --2- 172.21.15.143:0/2378637638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1056d0 0x7f9d7c105af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009990 tx=0x7f9d6c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 -- 172.21.15.143:0/2378637638 shutdown_connections 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 --2- 172.21.15.143:0/2378637638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 --2- 172.21.15.143:0/2378637638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1056d0 0x7f9d7c105af0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 --2- 172.21.15.143:0/2378637638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 -- 172.21.15.143:0/2378637638 >> 172.21.15.143:0/2378637638 conn(0x7f9d7c100c70 msgr2=0x7f9d7c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:58.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 -- 172.21.15.143:0/2378637638 shutdown_connections 2024-04-26T21:29:58.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.987+0000 7f9d83200700 1 -- 172.21.15.143:0/2378637638 wait complete. 2024-04-26T21:29:58.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.988+0000 7f9d83200700 1 Processor -- start 2024-04-26T21:29:58.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.988+0000 7f9d83200700 1 -- start start 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d7c1056d0 0x7f9d7c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c1a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c111040 con 0x7f9d7c107270 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c110ec0 con 0x7f9d7c1056d0 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d83200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1111c0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39690/0 (socket says 172.21.15.143:39690) 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 -- 172.21.15.143:0/2866916802 learned_addr learned my addr 172.21.15.143:0/2866916802 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 -- 172.21.15.143:0/2866916802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d7c1056d0 msgr2=0x7f9d7c19b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d7c1056d0 0x7f9d7c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d73e00700 1 -- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 msgr2=0x7f9d7c1a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:58.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.989+0000 7f9d81200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c1a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:58.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d73e00700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c1a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:58.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d73e00700 1 -- 172.21.15.143:0/2866916802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c1a33f0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d81200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c1a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:29:58.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d73e00700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c950 tx=0x7f9d6400cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:58.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64007ac0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d64004cc0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c1a36e0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64005570 con 0x7f9d7c1068d0 2024-04-26T21:29:58.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.990+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a3c20 con 0x7f9d7c1068d0 2024-04-26T21:29:58.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.991+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c06a110 con 0x7f9d7c1068d0 2024-04-26T21:29:58.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.994+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d64007cf0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.995+0000 7f9d79600700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:29:58.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.995+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f9d640cffe0 con 0x7f9d7c1068d0 2024-04-26T21:29:58.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.996+0000 7f9d7be00700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:29:58.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.997+0000 7f9d7be00700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0036f0 tx=0x7f9d6c005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:29:58.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:58.999+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9d64014030 con 0x7f9d7c1068d0 2024-04-26T21:29:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.238+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}) -- 0x7f9d7c063a60 con 0x7f9d600a3c40 2024-04-26T21:29:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.256+0000 7f9d79600700 1 -- 172.21.15.143:0/2866916802 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9d7c063a60 con 0x7f9d600a3c40 2024-04-26T21:29:59.257 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mds.all update... 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 msgr2=0x7f9d600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0036f0 tx=0x7f9d6c005c20 comp rx=0 tx=0).stop 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 msgr2=0x7f9d7c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c950 tx=0x7f9d6400cd10 comp rx=0 tx=0).stop 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 shutdown_connections 2024-04-26T21:29:59.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d7c1056d0 0x7f9d7c19b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d7c107270 0x7f9d7c1a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 --2- 172.21.15.143:0/2866916802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d7c1068d0 0x7f9d7c19bcd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.259+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 >> 172.21.15.143:0/2866916802 conn(0x7f9d7c100c70 msgr2=0x7f9d7c109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.260+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 shutdown_connections 2024-04-26T21:29:59.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-04-26T21:29:59.260+0000 7f9d83200700 1 -- 172.21.15.143:0/2866916802 wait complete. 2024-04-26T21:30:00.027 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:00.027 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.027 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:00.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:29:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:00.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:29:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.092 DEBUG:teuthology.orchestra.run.smithi097:mds.a> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:30:00.100 DEBUG:teuthology.orchestra.run.smithi097:mds.d> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:00.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:00.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:29:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:00.147 DEBUG:teuthology.orchestra.run.smithi097:mds.g> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:30:00.192 DEBUG:teuthology.orchestra.run.smithi097:mds.j> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:30:00.236 DEBUG:teuthology.orchestra.run.smithi111:mds.b> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:30:00.244 DEBUG:teuthology.orchestra.run.smithi111:mds.e> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:30:00.290 DEBUG:teuthology.orchestra.run.smithi111:mds.h> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:30:00.336 DEBUG:teuthology.orchestra.run.smithi111:mds.k> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:30:00.380 DEBUG:teuthology.orchestra.run.smithi143:mds.c> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:30:00.388 DEBUG:teuthology.orchestra.run.smithi143:mds.f> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:30:00.435 DEBUG:teuthology.orchestra.run.smithi143:mds.i> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:30:00.480 DEBUG:teuthology.orchestra.run.smithi143:mds.l> sudo journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:30:00.525 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:30:00.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 -- 172.21.15.97:0/4019278910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4102970 msgr2=0x7f52e4102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 --2- 172.21.15.97:0/4019278910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4102970 0x7f52e4102d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009900 tx=0x7f52d402a410 comp rx=0 tx=0).stop 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 -- 172.21.15.97:0/4019278910 shutdown_connections 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 --2- 172.21.15.97:0/4019278910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 0x7f52e410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 --2- 172.21.15.97:0/4019278910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4103b60 0x7f52e4103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 --2- 172.21.15.97:0/4019278910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4102970 0x7f52e4102d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 -- 172.21.15.97:0/4019278910 >> 172.21.15.97:0/4019278910 conn(0x7f52e40fdee0 msgr2=0x7f52e4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:00.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 -- 172.21.15.97:0/4019278910 shutdown_connections 2024-04-26T21:30:00.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.738+0000 7f52ebc00640 1 -- 172.21.15.97:0/4019278910 wait complete. 2024-04-26T21:30:00.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 Processor -- start 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 -- start start 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4102970 0x7f52e419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 0x7f52e41a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e340 con 0x7f52e4103b60 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e1c0 con 0x7f52e4104520 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.739+0000 7f52ebc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410e4c0 con 0x7f52e4102970 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46584/0 (socket says 172.21.15.97:46584) 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 -- 172.21.15.97:0/1882243396 learned_addr learned my addr 172.21.15.97:0/1882243396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52e9200640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4102970 0x7f52e419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52e9c00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 0x7f52e41a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 -- 172.21.15.97:0/1882243396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4102970 msgr2=0x7f52e419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4102970 0x7f52e419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 -- 172.21.15.97:0/1882243396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 msgr2=0x7f52e41a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 0x7f52e41a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 -- 172.21.15.97:0/1882243396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e41a2960 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.740+0000 7f52dbe00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00d770 tx=0x7f52cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.741+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc017070 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.741+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52cc0054b0 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.741+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41a2c50 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.741+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc004c60 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.741+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41a3150 con 0x7f52e4103b60 2024-04-26T21:30:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.742+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e41a3320 con 0x7f52e4103b60 2024-04-26T21:30:00.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.744+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52cc004e00 con 0x7f52e4103b60 2024-04-26T21:30:00.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.745+0000 7f52e2000640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 0x7f52c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:00.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.745+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f52cc0cfba0 con 0x7f52e4103b60 2024-04-26T21:30:00.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.746+0000 7f52e9200640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 0x7f52c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:00.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.746+0000 7f52e9200640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 0x7f52c80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52d40057c0 tx=0x7f52d40095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.750+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f52cc09c980 con 0x7f52e4103b60 2024-04-26T21:30:00.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:00 smithi143 ceph-mon[31461]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:30:00.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:00 smithi143 ceph-mon[31461]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-04-26T21:30:00.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:00 smithi143 ceph-mon[31461]: Deploying daemon mds.l on smithi143 2024-04-26T21:30:00.923 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:00 smithi143 ceph-mon[31461]: overall HEALTH_OK 2024-04-26T21:30:00.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.954+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52e41a3ac0 con 0x7f52e4103b60 2024-04-26T21:30:00.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.955+0000 7f52e2000640 1 -- 172.21.15.97:0/1882243396 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18457 (secure 0 0 0) 0x7f52cc09c280 con 0x7f52e4103b60 2024-04-26T21:30:00.957 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:29:48.485798+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 msgr2=0x7f52c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 0x7f52c80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52d40057c0 tx=0x7f52d40095f0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 msgr2=0x7f52e419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00d770 tx=0x7f52cc00dc30 comp rx=0 tx=0).stop 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 shutdown_connections 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52e4104520 0x7f52e41a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f52c80a3bb0 0x7f52c80a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e4103b60 0x7f52e419b290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 --2- 172.21.15.97:0/1882243396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52e4102970 0x7f52e419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 >> 172.21.15.97:0/1882243396 conn(0x7f52e40fdee0 msgr2=0x7f52e4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 shutdown_connections 2024-04-26T21:30:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:00.958+0000 7f52ebc00640 1 -- 172.21.15.97:0/1882243396 wait complete. 2024-04-26T21:30:00.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:00 smithi097 ceph-mon[28127]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:30:00.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:00 smithi097 ceph-mon[28127]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-04-26T21:30:00.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:00 smithi097 ceph-mon[28127]: Deploying daemon mds.l on smithi143 2024-04-26T21:30:00.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:00 smithi097 ceph-mon[28127]: overall HEALTH_OK 2024-04-26T21:30:00.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:30:00.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:30:01.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:00 smithi111 ceph-mon[31001]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:30:01.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:00 smithi111 ceph-mon[31001]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-04-26T21:30:01.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:00 smithi111 ceph-mon[31001]: Deploying daemon mds.l on smithi143 2024-04-26T21:30:01.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:00 smithi111 ceph-mon[31001]: overall HEALTH_OK 2024-04-26T21:30:01.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 -- 172.21.15.97:0/2966093309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac101740 msgr2=0x7f6eac101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 --2- 172.21.15.97:0/2966093309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac101740 0x7f6eac101b60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009900 tx=0x7f6e9c02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:01.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 -- 172.21.15.97:0/2966093309 shutdown_connections 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 --2- 172.21.15.97:0/2966093309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac1032f0 0x7f6eac109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 --2- 172.21.15.97:0/2966093309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac102930 0x7f6eac102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 --2- 172.21.15.97:0/2966093309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac101740 0x7f6eac101b60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 -- 172.21.15.97:0/2966093309 >> 172.21.15.97:0/2966093309 conn(0x7f6eac0fccd0 msgr2=0x7f6eac0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 -- 172.21.15.97:0/2966093309 shutdown_connections 2024-04-26T21:30:01.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.185+0000 7f6eabe00640 1 -- 172.21.15.97:0/2966093309 wait complete. 2024-04-26T21:30:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.186+0000 7f6eabe00640 1 Processor -- start 2024-04-26T21:30:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.186+0000 7f6eabe00640 1 -- start start 2024-04-26T21:30:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac101740 0x7f6eac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac1032f0 0x7f6eac1a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10d170 con 0x7f6eac102930 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10cff0 con 0x7f6eac1032f0 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eabe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10d2f0 con 0x7f6eac101740 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6ea2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6ea2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36532/0 (socket says 172.21.15.97:36532) 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6ea2000640 1 -- 172.21.15.97:0/4179102350 learned_addr learned my addr 172.21.15.97:0/4179102350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:01.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eaaa00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac101740 0x7f6eac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.187+0000 7f6eab400640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac1032f0 0x7f6eac1a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 -- 172.21.15.97:0/4179102350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac101740 msgr2=0x7f6eac19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac101740 0x7f6eac19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 -- 172.21.15.97:0/4179102350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac1032f0 msgr2=0x7f6eac1a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac1032f0 0x7f6eac1a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 -- 172.21.15.97:0/4179102350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a2850 con 0x7f6eac102930 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6ea2000640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d950 tx=0x7f6e9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94017070 con 0x7f6eac102930 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e94009c30 con 0x7f6eac102930 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400f620 con 0x7f6eac102930 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.188+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a2b40 con 0x7f6eac102930 2024-04-26T21:30:01.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.189+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a30b0 con 0x7f6eac102930 2024-04-26T21:30:01.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.190+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac1a3280 con 0x7f6eac102930 2024-04-26T21:30:01.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.192+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e94010460 con 0x7f6eac102930 2024-04-26T21:30:01.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.193+0000 7f6eb0a00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 0x7f6e900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.193+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6e940cfcd0 con 0x7f6eac102930 2024-04-26T21:30:01.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.193+0000 7f6eaaa00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 0x7f6e900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.194+0000 7f6eaaa00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 0x7f6e900a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0057c0 tx=0x7f6e9c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.197+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6e9409cab0 con 0x7f6eac102930 2024-04-26T21:30:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.402+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6eac1a3610 con 0x7f6eac102930 2024-04-26T21:30:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.403+0000 7f6eb0a00640 1 -- 172.21.15.97:0/4179102350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f6e94009de0 con 0x7f6eac102930 2024-04-26T21:30:01.404 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 msgr2=0x7f6e900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 0x7f6e900a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0057c0 tx=0x7f6e9c031400 comp rx=0 tx=0).stop 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 msgr2=0x7f6eac19b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d950 tx=0x7f6e9400de10 comp rx=0 tx=0).stop 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 shutdown_connections 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eac1032f0 0x7f6eac1a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6e900a3c30 0x7f6e900a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac102930 0x7f6eac19b180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 --2- 172.21.15.97:0/4179102350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eac101740 0x7f6eac19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 >> 172.21.15.97:0/4179102350 conn(0x7f6eac0fccd0 msgr2=0x7f6eac069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 shutdown_connections 2024-04-26T21:30:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.406+0000 7f6eabe00640 1 -- 172.21.15.97:0/4179102350 wait complete. 2024-04-26T21:30:01.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.627+0000 7f853ee00640 1 -- 172.21.15.97:0/522660040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 msgr2=0x7f85381093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.627+0000 7f853ee00640 1 --2- 172.21.15.97:0/522660040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f85381093b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 -- 172.21.15.97:0/522660040 shutdown_connections 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 --2- 172.21.15.97:0/522660040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 0x7f85381113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 --2- 172.21.15.97:0/522660040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 --2- 172.21.15.97:0/522660040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f85381093b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 -- 172.21.15.97:0/522660040 >> 172.21.15.97:0/522660040 conn(0x7f8538077f40 msgr2=0x7f853807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 -- 172.21.15.97:0/522660040 shutdown_connections 2024-04-26T21:30:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.628+0000 7f853ee00640 1 -- 172.21.15.97:0/522660040 wait complete. 2024-04-26T21:30:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 Processor -- start 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 -- start start 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 0x7f85381a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85381149d0 con 0x7f8538108f90 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8538114850 con 0x7f853810a180 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f853ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8538114b50 con 0x7f853810ab40 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f8537e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f8537e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36564/0 (socket says 172.21.15.97:36564) 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.629+0000 7f8537e00640 1 -- 172.21.15.97:0/2458100716 learned_addr learned my addr 172.21.15.97:0/2458100716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f853ce00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 0x7f85381a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537400640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 -- 172.21.15.97:0/2458100716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 msgr2=0x7f85381a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 0x7f85381a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 -- 172.21.15.97:0/2458100716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 msgr2=0x7f853819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 -- 172.21.15.97:0/2458100716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381a6cd0 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537400640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8537e00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8528031460 tx=0x7f8528003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852803f070 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.630+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8528031da0 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.631+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85381a6f60 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.631+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528035030 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.631+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85381a7370 con 0x7f8538108f90 2024-04-26T21:30:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.632+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8538110350 con 0x7f8538108f90 2024-04-26T21:30:01.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.634+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8528042070 con 0x7f8538108f90 2024-04-26T21:30:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.635+0000 7f8534c00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 0x7f851c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.635+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f85280f0950 con 0x7f8538108f90 2024-04-26T21:30:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.635+0000 7f8537400640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 0x7f851c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.636+0000 7f8537400640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 0x7f851c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85381a05c0 tx=0x7f8524005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:01.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.639+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f85280bd730 con 0x7f8538108f90 2024-04-26T21:30:01.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.888+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f853802dad0 con 0x7f8538108f90 2024-04-26T21:30:01.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.890+0000 7f8534c00640 1 -- 172.21.15.97:0/2458100716 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f85280bd030 con 0x7f8538108f90 2024-04-26T21:30:01.891 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:01.891 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:30:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 1 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.892+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 msgr2=0x7f851c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.892+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 0x7f851c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85381a05c0 tx=0x7f8524005e50 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 msgr2=0x7f853819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8528031460 tx=0x7f8528003c80 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 shutdown_connections 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f853810ab40 0x7f85381a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f853810a180 0x7f853819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f851c0a3bb0 0x7f851c0a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 --2- 172.21.15.97:0/2458100716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108f90 0x7f853819f0b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 >> 172.21.15.97:0/2458100716 conn(0x7f8538077f40 msgr2=0x7f853810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:01.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 shutdown_connections 2024-04-26T21:30:01.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:01.893+0000 7f853ee00640 1 -- 172.21.15.97:0/2458100716 wait complete. 2024-04-26T21:30:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:01 smithi143 ceph-mon[31461]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:02.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:01 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1882243396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:02.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:01 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4179102350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:01 smithi097 ceph-mon[28127]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:01 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1882243396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:02.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:01 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4179102350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:01 smithi111 ceph-mon[31001]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:01 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1882243396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:01 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4179102350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:02.730 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:02 smithi143 systemd[1]: Starting Ceph mds.l for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:02 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2458100716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:03.045 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:02 smithi143 podman[61144]: 2024-04-26 21:30:02.687515664 +0000 UTC m=+0.032871185 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:03.045 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:02 smithi143 podman[61144]: 2024-04-26 21:30:02.803422219 +0000 UTC m=+0.148777778 container create 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T21:30:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:02 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2458100716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:02 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2458100716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:03.546 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 podman[61144]: 2024-04-26 21:30:03.066091074 +0000 UTC m=+0.411446640 container init 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:30:03.546 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 podman[61144]: 2024-04-26 21:30:03.076172718 +0000 UTC m=+0.421528247 container start 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:30:03.546 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mds[61163]: -- 172.21.15.143:0/598431113 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a724d5da40 con 0x55a724d30800 2024-04-26T21:30:03.546 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: starting mds.l at 2024-04-26T21:30:03.547 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 bash[61144]: 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 2024-04-26T21:30:03.547 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:30:03 smithi143 systemd[1]: Started Ceph mds.l for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:03.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:03.992 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6832/845773802,v1:172.21.15.143:6833/845773802] up:boot 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: fsmap 1 up:standby 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:03.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:03 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6832/845773802,v1:172.21.15.143:6833/845773802] up:boot 2024-04-26T21:30:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: fsmap 1 up:standby 2024-04-26T21:30:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:04.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:03 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 -- 172.21.15.97:0/1196909594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58108d90 msgr2=0x7f8e581091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/1196909594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58108d90 0x7f8e581091b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 -- 172.21.15.97:0/1196909594 shutdown_connections 2024-04-26T21:30:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/1196909594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5810a940 0x7f8e581111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/1196909594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58109f80 0x7f8e5810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/1196909594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58108d90 0x7f8e581091b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 -- 172.21.15.97:0/1196909594 >> 172.21.15.97:0/1196909594 conn(0x7f8e58077d30 msgr2=0x7f8e5807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 -- 172.21.15.97:0/1196909594 shutdown_connections 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.125+0000 7f8e5ee00640 1 -- 172.21.15.97:0/1196909594 wait complete. 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 Processor -- start 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 -- start start 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58108d90 0x7f8e5819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e58109f80 0x7f8e5819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58114790 con 0x7f8e5810a940 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58114610 con 0x7f8e58109f80 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.126+0000 7f8e5ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58114910 con 0x7f8e58108d90 2024-04-26T21:30:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36576/0 (socket says 172.21.15.97:36576) 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 -- 172.21.15.97:0/3462277323 learned_addr learned my addr 172.21.15.97:0/3462277323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5d000640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e58109f80 0x7f8e5819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5da00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58108d90 0x7f8e5819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 -- 172.21.15.97:0/3462277323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58108d90 msgr2=0x7f8e5819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58108d90 0x7f8e5819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 -- 172.21.15.97:0/3462277323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e58109f80 msgr2=0x7f8e5819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e58109f80 0x7f8e5819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.127+0000 7f8e5e400640 1 -- 172.21.15.97:0/3462277323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581a6a70 con 0x7f8e5810a940 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e5e400640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eaa0 tx=0x7f8e5000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004d20 con 0x7f8e5810a940 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581a6d60 con 0x7f8e5810a940 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e50005020 con 0x7f8e5810a940 2024-04-26T21:30:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500116d0 con 0x7f8e5810a940 2024-04-26T21:30:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.128+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a7200 con 0x7f8e5810a940 2024-04-26T21:30:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.129+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e58110180 con 0x7f8e5810a940 2024-04-26T21:30:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.131+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e500051d0 con 0x7f8e5810a940 2024-04-26T21:30:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.133+0000 7f8e56000640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.134+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8e500d0100 con 0x7f8e5810a940 2024-04-26T21:30:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.134+0000 7f8e5da00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.134+0000 7f8e5da00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c031460 tx=0x7f8e4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.138+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e50016030 con 0x7f8e5810a940 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6832/845773802,v1:172.21.15.143:6833/845773802] up:boot 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: fsmap 1 up:standby 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:03 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.380+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5802dad0 con 0x7f8e5810a940 2024-04-26T21:30:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.381+0000 7f8e56000640 1 -- 172.21.15.97:0/3462277323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f8e5009cf80 con 0x7f8e5810a940 2024-04-26T21:30:04.382 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:04.382 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:30:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.383+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 msgr2=0x7f8e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.383+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c031460 tx=0x7f8e4c005fd0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.383+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 msgr2=0x7f8e581a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.383+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eaa0 tx=0x7f8e5000eea0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 shutdown_connections 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e5810a940 0x7f8e581a6400 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e58109f80 0x7f8e5819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 --2- 172.21.15.97:0/3462277323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e58108d90 0x7f8e5819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 >> 172.21.15.97:0/3462277323 conn(0x7f8e58077d30 msgr2=0x7f8e5810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 shutdown_connections 2024-04-26T21:30:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:04.384+0000 7f8e5ee00640 1 -- 172.21.15.97:0/3462277323 wait complete. 2024-04-26T21:30:04.987 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:04 smithi143 ceph-mon[31461]: Deploying daemon mds.k on smithi111 2024-04-26T21:30:04.987 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:04 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3462277323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:05.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:04 smithi097 ceph-mon[28127]: Deploying daemon mds.k on smithi111 2024-04-26T21:30:05.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:04 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3462277323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:04 smithi111 ceph-mon[31001]: Deploying daemon mds.k on smithi111 2024-04-26T21:30:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:04 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3462277323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:05.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:05 smithi111 ceph-mon[31001]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:05 smithi143 ceph-mon[31461]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:06.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:05 smithi097 ceph-mon[28127]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:06.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:06.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 -- 172.21.15.97:0/3914351453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7688109f80 msgr2=0x7f768810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 --2- 172.21.15.97:0/3914351453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7688109f80 0x7f768810a400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7670009900 tx=0x7f767002a410 comp rx=0 tx=0).stop 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 -- 172.21.15.97:0/3914351453 shutdown_connections 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 --2- 172.21.15.97:0/3914351453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f768810a940 0x7f76881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 --2- 172.21.15.97:0/3914351453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7688109f80 0x7f768810a400 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 --2- 172.21.15.97:0/3914351453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 0x7f76881091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.615+0000 7f7687200640 1 -- 172.21.15.97:0/3914351453 >> 172.21.15.97:0/3914351453 conn(0x7f7688077d30 msgr2=0x7f768807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.616+0000 7f7687200640 1 -- 172.21.15.97:0/3914351453 shutdown_connections 2024-04-26T21:30:06.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.616+0000 7f7687200640 1 -- 172.21.15.97:0/3914351453 wait complete. 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 Processor -- start 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 -- start start 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 0x7f768819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 0x7f768819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7688114840 con 0x7f768810a940 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f76881146c0 con 0x7f7688109f80 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7687200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f76881149c0 con 0x7f7688108d90 2024-04-26T21:30:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.617+0000 7f7686800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36588/0 (socket says 172.21.15.97:36588) 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 -- 172.21.15.97:0/3734423185 learned_addr learned my addr 172.21.15.97:0/3734423185 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7685e00640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 0x7f768819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f767d400640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 0x7f768819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 -- 172.21.15.97:0/3734423185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 msgr2=0x7f768819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 0x7f768819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 -- 172.21.15.97:0/3734423185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 msgr2=0x7f768819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 0x7f768819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 -- 172.21.15.97:0/3734423185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76881a6b60 con 0x7f768810a940 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f767d400640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 0x7f768819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f7686800640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f768000b970 tx=0x7f768000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680019070 con 0x7f768810a940 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7680014440 con 0x7f768810a940 2024-04-26T21:30:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.618+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800075d0 con 0x7f768810a940 2024-04-26T21:30:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.619+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76881a6e50 con 0x7f768810a940 2024-04-26T21:30:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.619+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a7340 con 0x7f768810a940 2024-04-26T21:30:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.620+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7688110100 con 0x7f768810a940 2024-04-26T21:30:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.622+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f768001c070 con 0x7f768810a940 2024-04-26T21:30:06.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.623+0000 7f767f400640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 0x7f766c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:06.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.624+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f76800cf580 con 0x7f768810a940 2024-04-26T21:30:06.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.624+0000 7f7685e00640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 0x7f766c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.624+0000 7f7685e00640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 0x7f766c0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7678005fd0 tx=0x7f7678005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:06.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.628+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f768009c360 con 0x7f768810a940 2024-04-26T21:30:06.643 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:06 smithi111 systemd[1]: Starting Ceph mds.k for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:06.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.871+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f768802dad0 con 0x7f768810a940 2024-04-26T21:30:06.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.871+0000 7f767f400640 1 -- 172.21.15.97:0/3734423185 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f76800bae20 con 0x7f768810a940 2024-04-26T21:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:30:06.873 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-04-26T21:30:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 msgr2=0x7f766c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 0x7f766c0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7678005fd0 tx=0x7f7678005850 comp rx=0 tx=0).stop 2024-04-26T21:30:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 msgr2=0x7f76881a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f768000b970 tx=0x7f768000be30 comp rx=0 tx=0).stop 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 shutdown_connections 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f766c0a3c30 0x7f766c0a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f768810a940 0x7f76881a64f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7688109f80 0x7f768819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 --2- 172.21.15.97:0/3734423185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7688108d90 0x7f768819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.874+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 >> 172.21.15.97:0/3734423185 conn(0x7f7688077d30 msgr2=0x7f768810d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.875+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 shutdown_connections 2024-04-26T21:30:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:06.875+0000 7f7687200640 1 -- 172.21.15.97:0/3734423185 wait complete. 2024-04-26T21:30:06.954 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:06 smithi111 podman[60308]: 2024-04-26 21:30:06.56572831 +0000 UTC m=+0.036523343 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:06.954 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:06 smithi111 podman[60308]: 2024-04-26 21:30:06.681390843 +0000 UTC m=+0.152185914 container create 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T21:30:06.954 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:06 smithi111 podman[60308]: 2024-04-26 21:30:06.952596738 +0000 UTC m=+0.423391789 container init 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T21:30:07.392 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:06 smithi111 podman[60308]: 2024-04-26 21:30:06.96433684 +0000 UTC m=+0.435131867 container start 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:30:07.393 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mds[60326]: -- 172.21.15.111:0/2703207341 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559299dada40 con 0x559299d80c00 2024-04-26T21:30:07.393 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: starting mds.k at 2024-04-26T21:30:07.393 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:07 smithi111 bash[60308]: 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 2024-04-26T21:30:07.393 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:30:07 smithi111 systemd[1]: Started Ceph mds.k for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3734423185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:07 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3734423185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:08.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:07 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3734423185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:08.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:08.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:08.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:07 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:08.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:08 smithi097 ceph-mon[28127]: Deploying daemon mds.j on smithi097 2024-04-26T21:30:08.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:08 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:boot 2024-04-26T21:30:08.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:08 smithi097 ceph-mon[28127]: fsmap 2 up:standby 2024-04-26T21:30:08.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:08 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:30:08.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:09.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:08 smithi143 ceph-mon[31461]: Deploying daemon mds.j on smithi097 2024-04-26T21:30:09.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:08 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:boot 2024-04-26T21:30:09.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:08 smithi143 ceph-mon[31461]: fsmap 2 up:standby 2024-04-26T21:30:09.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:08 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:30:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 -- 172.21.15.97:0/3136845244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 msgr2=0x7f2e38104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 --2- 172.21.15.97:0/3136845244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e38104dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009900 tx=0x7f2e2802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 -- 172.21.15.97:0/3136845244 shutdown_connections 2024-04-26T21:30:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 --2- 172.21.15.97:0/3136845244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 0x7f2e3810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 --2- 172.21.15.97:0/3136845244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 0x7f2e38106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 --2- 172.21.15.97:0/3136845244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e38104dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 -- 172.21.15.97:0/3136845244 >> 172.21.15.97:0/3136845244 conn(0x7f2e380fff40 msgr2=0x7f2e38102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 -- 172.21.15.97:0/3136845244 shutdown_connections 2024-04-26T21:30:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.114+0000 7f2e3e000640 1 -- 172.21.15.97:0/3136845244 wait complete. 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.115+0000 7f2e3e000640 1 Processor -- start 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.115+0000 7f2e3e000640 1 -- start start 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 0x7f2e3819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 0x7f2e381a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e38110390 con 0x7f2e381049b0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e38110210 con 0x7f2e38105ba0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2e38110510 con 0x7f2e38106560 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36610/0 (socket says 172.21.15.97:36610) 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e3cc00640 1 -- 172.21.15.97:0/4202451719 learned_addr learned my addr 172.21.15.97:0/4202451719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.116+0000 7f2e2fe00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 0x7f2e3819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3d600640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 0x7f2e381a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 -- 172.21.15.97:0/4202451719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 msgr2=0x7f2e381a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 0x7f2e381a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 -- 172.21.15.97:0/4202451719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 msgr2=0x7f2e3819af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 0x7f2e3819af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 -- 172.21.15.97:0/4202451719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e381a2660 con 0x7f2e381049b0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3cc00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005ab0 tx=0x7f2e280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2803f030 con 0x7f2e381049b0 2024-04-26T21:30:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e28003c90 con 0x7f2e381049b0 2024-04-26T21:30:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e381a28f0 con 0x7f2e381049b0 2024-04-26T21:30:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28035070 con 0x7f2e381049b0 2024-04-26T21:30:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.117+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e381a2de0 con 0x7f2e381049b0 2024-04-26T21:30:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.119+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3810bd50 con 0x7f2e381049b0 2024-04-26T21:30:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.121+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e28042030 con 0x7f2e381049b0 2024-04-26T21:30:09.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.122+0000 7f2e36000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 0x7f2e1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:09.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.122+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f2e280f0a70 con 0x7f2e381049b0 2024-04-26T21:30:09.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.122+0000 7f2e2fe00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 0x7f2e1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:09.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.123+0000 7f2e2fe00640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 0x7f2e1c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005e10 tx=0x7f2e20005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:09.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.126+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e280bd850 con 0x7f2e381049b0 2024-04-26T21:30:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:08 smithi111 ceph-mon[31001]: Deploying daemon mds.j on smithi097 2024-04-26T21:30:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:08 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:boot 2024-04-26T21:30:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:08 smithi111 ceph-mon[31001]: fsmap 2 up:standby 2024-04-26T21:30:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:08 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:30:09.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.373+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e3802dcf0 con 0x7f2e381049b0 2024-04-26T21:30:09.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.374+0000 7f2e36000640 1 -- 172.21.15.97:0/4202451719 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f2e280bd150 con 0x7f2e381049b0 2024-04-26T21:30:09.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:09.375 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:30:09.376 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 msgr2=0x7f2e1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 0x7f2e1c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005e10 tx=0x7f2e20005da0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 msgr2=0x7f2e3819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005ab0 tx=0x7f2e280042f0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 shutdown_connections 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e38106560 0x7f2e381a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38105ba0 0x7f2e3819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e1c0a3bb0 0x7f2e1c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 --2- 172.21.15.97:0/4202451719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e381049b0 0x7f2e3819aa40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 >> 172.21.15.97:0/4202451719 conn(0x7f2e380fff40 msgr2=0x7f2e38108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 shutdown_connections 2024-04-26T21:30:09.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:09.377+0000 7f2e3e000640 1 -- 172.21.15.97:0/4202451719 wait complete. 2024-04-26T21:30:09.995 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:09 smithi097 ceph-mon[28127]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:09.995 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:09 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4202451719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:09 smithi143 ceph-mon[31461]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:09 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4202451719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:09 smithi111 ceph-mon[31001]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:09 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4202451719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:10.564 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 systemd[1]: Starting Ceph mds.j for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:10.908 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 podman[68355]: 2024-04-26 21:30:10.627757061 +0000 UTC m=+0.120199731 container create 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T21:30:10.909 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 podman[68355]: 2024-04-26 21:30:10.541841906 +0000 UTC m=+0.034284545 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:10.909 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 podman[68355]: 2024-04-26 21:30:10.907066415 +0000 UTC m=+0.399509092 container init 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:30:11.314 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 podman[68355]: 2024-04-26 21:30:10.918539033 +0000 UTC m=+0.410981673 container start 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:30:11.314 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 ceph-mds[68373]: -- 172.21.15.97:0/1769211554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b8f74fa40 con 0x563b8f723000 2024-04-26T21:30:11.314 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j[68369]: starting mds.j at 2024-04-26T21:30:11.314 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:11 smithi097 bash[68355]: 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 2024-04-26T21:30:11.314 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:30:11 smithi097 systemd[1]: Started Ceph mds.j for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:11.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 -- 172.21.15.97:0/1413296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc102770 msgr2=0x7f07fc102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 --2- 172.21.15.97:0/1413296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc102770 0x7f07fc102b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009900 tx=0x7f07e402a410 comp rx=0 tx=0).stop 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 -- 172.21.15.97:0/1413296511 shutdown_connections 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 --2- 172.21.15.97:0/1413296511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc104320 0x7f07fc10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 --2- 172.21.15.97:0/1413296511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 0x7f07fc103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 --2- 172.21.15.97:0/1413296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc102770 0x7f07fc102b90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 -- 172.21.15.97:0/1413296511 >> 172.21.15.97:0/1413296511 conn(0x7f07fc0fdcc0 msgr2=0x7f07fc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.601+0000 7f0801c00640 1 -- 172.21.15.97:0/1413296511 shutdown_connections 2024-04-26T21:30:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.602+0000 7f0801c00640 1 -- 172.21.15.97:0/1413296511 wait complete. 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.602+0000 7f0801c00640 1 Processor -- start 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 -- start start 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 0x7f07fc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 0x7f07fc19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10e190 con 0x7f07fc104320 2024-04-26T21:30:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10e010 con 0x7f07fc102770 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10e310 con 0x7f07fc103960 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50302/0 (socket says 172.21.15.97:50302) 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f0801200640 1 -- 172.21.15.97:0/3711715322 learned_addr learned my addr 172.21.15.97:0/3711715322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.603+0000 7f07fbe00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 0x7f07fc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f07f3400640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 0x7f07fc19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 -- 172.21.15.97:0/3711715322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 msgr2=0x7f07fc19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 0x7f07fc19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 -- 172.21.15.97:0/3711715322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 msgr2=0x7f07fc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 0x7f07fc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 -- 172.21.15.97:0/3711715322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc1a2640 con 0x7f07fc104320 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f07fbe00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 0x7f07fc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801200640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07f400cce0 tx=0x7f07f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f400f450 con 0x7f07fc104320 2024-04-26T21:30:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07f400fa80 con 0x7f07fc104320 2024-04-26T21:30:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018ad0 con 0x7f07fc104320 2024-04-26T21:30:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.604+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc1a2930 con 0x7f07fc104320 2024-04-26T21:30:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.605+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc1a2e30 con 0x7f07fc104320 2024-04-26T21:30:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.606+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07fc1a30c0 con 0x7f07fc104320 2024-04-26T21:30:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.608+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07f400f5f0 con 0x7f07fc104320 2024-04-26T21:30:11.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.609+0000 7f07f9600640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 0x7f07e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:11.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.609+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f07f40d0810 con 0x7f07fc104320 2024-04-26T21:30:11.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.609+0000 7f07fbe00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 0x7f07e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:11.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.610+0000 7f07fbe00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 0x7f07e00a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07e40057c0 tx=0x7f07e4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.619+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07f4014030 con 0x7f07fc104320 2024-04-26T21:30:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.867+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07fc1a3290 con 0x7f07fc104320 2024-04-26T21:30:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.867+0000 7f07f9600640 1 -- 172.21.15.97:0/3711715322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f07f409d5f0 con 0x7f07fc104320 2024-04-26T21:30:11.868 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:11.869 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:30:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-04-26T21:30:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 msgr2=0x7f07e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 0x7f07e00a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07e40057c0 tx=0x7f07e4031400 comp rx=0 tx=0).stop 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 msgr2=0x7f07fc1a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07f400cce0 tx=0x7f07f400a620 comp rx=0 tx=0).stop 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 shutdown_connections 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f07e00a3c30 0x7f07e00a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc104320 0x7f07fc1a1fd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07fc103960 0x7f07fc19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 --2- 172.21.15.97:0/3711715322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f07fc102770 0x7f07fc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 >> 172.21.15.97:0/3711715322 conn(0x7f07fc0fdcc0 msgr2=0x7f07fc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 shutdown_connections 2024-04-26T21:30:11.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:11.870+0000 7f0801c00640 1 -- 172.21.15.97:0/3711715322 wait complete. 2024-04-26T21:30:12.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:11 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:12.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:12.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:11 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:12.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:11 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:12 smithi097 ceph-mon[28127]: Deploying daemon mds.d on smithi097 2024-04-26T21:30:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:12 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6834/4203195871,v1:172.21.15.97:6835/4203195871] up:boot 2024-04-26T21:30:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:12 smithi097 ceph-mon[28127]: fsmap 3 up:standby 2024-04-26T21:30:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:12 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:30:13.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:12 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3711715322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:12 smithi111 ceph-mon[31001]: Deploying daemon mds.d on smithi097 2024-04-26T21:30:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:12 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6834/4203195871,v1:172.21.15.97:6835/4203195871] up:boot 2024-04-26T21:30:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:12 smithi111 ceph-mon[31001]: fsmap 3 up:standby 2024-04-26T21:30:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:12 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:30:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:12 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3711715322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:13.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:12 smithi143 ceph-mon[31461]: Deploying daemon mds.d on smithi097 2024-04-26T21:30:13.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:12 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6834/4203195871,v1:172.21.15.97:6835/4203195871] up:boot 2024-04-26T21:30:13.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:12 smithi143 ceph-mon[31461]: fsmap 3 up:standby 2024-04-26T21:30:13.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:12 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:30:13.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:12 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3711715322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:13.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:13.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:13 smithi111 ceph-mon[31001]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:14.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:13 smithi097 ceph-mon[28127]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:14.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:13 smithi143 ceph-mon[31461]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 -- 172.21.15.97:0/2796921711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74105d80 msgr2=0x7f7e74106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 --2- 172.21.15.97:0/2796921711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74105d80 0x7f7e74106200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a410 comp rx=0 tx=0).stop 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 -- 172.21.15.97:0/2796921711 shutdown_connections 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 --2- 172.21.15.97:0/2796921711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74106740 0x7f7e7410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 --2- 172.21.15.97:0/2796921711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74105d80 0x7f7e74106200 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 --2- 172.21.15.97:0/2796921711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 0x7f7e74104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 -- 172.21.15.97:0/2796921711 >> 172.21.15.97:0/2796921711 conn(0x7f7e74100120 msgr2=0x7f7e74102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 -- 172.21.15.97:0/2796921711 shutdown_connections 2024-04-26T21:30:14.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.307+0000 7f7e7a400640 1 -- 172.21.15.97:0/2796921711 wait complete. 2024-04-26T21:30:14.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.308+0000 7f7e7a400640 1 Processor -- start 2024-04-26T21:30:14.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.308+0000 7f7e7a400640 1 -- start start 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 0x7f7e7419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74105d80 0x7f7e7419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741105c0 con 0x7f7e74106740 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74110440 con 0x7f7e74104b90 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e7a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74110740 con 0x7f7e74105d80 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50326/0 (socket says 172.21.15.97:50326) 2024-04-26T21:30:14.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e73e00640 1 -- 172.21.15.97:0/703564269 learned_addr learned my addr 172.21.15.97:0/703564269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:14.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e73400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 0x7f7e7419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:14.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.309+0000 7f7e72a00640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74105d80 0x7f7e7419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 -- 172.21.15.97:0/703564269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74105d80 msgr2=0x7f7e7419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74105d80 0x7f7e7419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 -- 172.21.15.97:0/703564269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 msgr2=0x7f7e7419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 0x7f7e7419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 -- 172.21.15.97:0/703564269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e741a2900 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e73e00640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00eaa0 tx=0x7f7e6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00cdb0 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e6c005020 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e741a2bf0 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0116d0 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.310+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a3110 con 0x7f7e74106740 2024-04-26T21:30:14.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.312+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7410bf30 con 0x7f7e74106740 2024-04-26T21:30:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.313+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e6c0051d0 con 0x7f7e74106740 2024-04-26T21:30:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.315+0000 7f7e6be00640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 0x7f7e580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.315+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7e6c0d01c0 con 0x7f7e74106740 2024-04-26T21:30:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.315+0000 7f7e73400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 0x7f7e580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.316+0000 7f7e73400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 0x7f7e580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419bb90 tx=0x7f7e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:14.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.320+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e6c016030 con 0x7f7e74106740 2024-04-26T21:30:14.564 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 systemd[1]: Starting Ceph mds.d for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.569+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e7402dad0 con 0x7f7e74106740 2024-04-26T21:30:14.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.571+0000 7f7e6be00640 1 -- 172.21.15.97:0/703564269 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f7e6c09cfa0 con 0x7f7e74106740 2024-04-26T21:30:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:14.572 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:30:14.572 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 msgr2=0x7f7e580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 0x7f7e580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419bb90 tx=0x7f7e64005f90 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 msgr2=0x7f7e741a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00eaa0 tx=0x7f7e6c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 shutdown_connections 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e580a3bb0 0x7f7e580a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.573+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e74106740 0x7f7e741a2260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.574+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e74105d80 0x7f7e7419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.574+0000 7f7e7a400640 1 --2- 172.21.15.97:0/703564269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e74104b90 0x7f7e7419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.574+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 >> 172.21.15.97:0/703564269 conn(0x7f7e74100120 msgr2=0x7f7e74109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.574+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 shutdown_connections 2024-04-26T21:30:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:14.574+0000 7f7e7a400640 1 -- 172.21.15.97:0/703564269 wait complete. 2024-04-26T21:30:14.835 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 podman[68730]: 2024-04-26 21:30:14.469102057 +0000 UTC m=+0.032113613 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:14.835 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 podman[68730]: 2024-04-26 21:30:14.574413627 +0000 UTC m=+0.137425211 container create f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T21:30:15.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:14 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/703564269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:15.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:14 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/703564269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:14 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/703564269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 podman[68730]: 2024-04-26 21:30:14.837281857 +0000 UTC m=+0.400293484 container init f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 podman[68730]: 2024-04-26 21:30:14.847378091 +0000 UTC m=+0.410389701 container start f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 ceph-mds[68750]: -- 172.21.15.97:0/934181707 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2100eba40 con 0x55d2100bec00 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d[68746]: starting mds.d at 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 bash[68730]: f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b 2024-04-26T21:30:15.315 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:30:14 smithi097 systemd[1]: Started Ceph mds.d for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:16.103 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:15 smithi111 ceph-mon[31001]: Deploying daemon mds.e on smithi111 2024-04-26T21:30:16.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:16.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:16.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:16.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:16.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:15 smithi143 ceph-mon[31461]: Deploying daemon mds.e on smithi111 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:15 smithi097 ceph-mon[28127]: Deploying daemon mds.e on smithi111 2024-04-26T21:30:16.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:16.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 -- 172.21.15.97:0/3222288559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 msgr2=0x7fab68107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:16.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 --2- 172.21.15.97:0/3222288559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab68107f10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fab50009900 tx=0x7fab5002a410 comp rx=0 tx=0).stop 2024-04-26T21:30:16.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 -- 172.21.15.97:0/3222288559 shutdown_connections 2024-04-26T21:30:16.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 --2- 172.21.15.97:0/3222288559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab68108450 0x7fab6810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:16.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 --2- 172.21.15.97:0/3222288559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab68107f10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:16.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 --2- 172.21.15.97:0/3222288559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab681031f0 0x7fab681055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:16.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 -- 172.21.15.97:0/3222288559 >> 172.21.15.97:0/3222288559 conn(0x7fab680fcbc0 msgr2=0x7fab680ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:16.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 -- 172.21.15.97:0/3222288559 shutdown_connections 2024-04-26T21:30:16.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.813+0000 7fab6d800640 1 -- 172.21.15.97:0/3222288559 wait complete. 2024-04-26T21:30:16.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.814+0000 7fab6d800640 1 Processor -- start 2024-04-26T21:30:16.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.814+0000 7fab6d800640 1 -- start start 2024-04-26T21:30:16.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab681031f0 0x7fab6819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:16.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:16.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab68108450 0x7fab681a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab6810e160 con 0x7fab68105b20 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab6810dfe0 con 0x7fab681031f0 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6d800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab6810e2e0 con 0x7fab68108450 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50366/0 (socket says 172.21.15.97:50366) 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 -- 172.21.15.97:0/3278490762 learned_addr learned my addr 172.21.15.97:0/3278490762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab67e00640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab681031f0 0x7fab6819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 -- 172.21.15.97:0/3278490762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab68108450 msgr2=0x7fab681a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab6ce00640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab68108450 0x7fab681a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab68108450 0x7fab681a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 -- 172.21.15.97:0/3278490762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab681031f0 msgr2=0x7fab6819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab681031f0 0x7fab6819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:16.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.815+0000 7fab5f400640 1 -- 172.21.15.97:0/3278490762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab681a2730 con 0x7fab68105b20 2024-04-26T21:30:16.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab5f400640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab50005ab0 tx=0x7fab500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:16.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5003f030 con 0x7fab68105b20 2024-04-26T21:30:16.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab50003c90 con 0x7fab68105b20 2024-04-26T21:30:16.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab681a29c0 con 0x7fab68105b20 2024-04-26T21:30:16.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50035070 con 0x7fab68105b20 2024-04-26T21:30:16.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.816+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab681a2eb0 con 0x7fab68105b20 2024-04-26T21:30:16.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.817+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68069670 con 0x7fab68105b20 2024-04-26T21:30:16.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.819+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab50042030 con 0x7fab68105b20 2024-04-26T21:30:16.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.821+0000 7fab65600640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 0x7fab4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.821+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fab500f0a70 con 0x7fab68105b20 2024-04-26T21:30:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.821+0000 7fab67e00640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 0x7fab4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:16.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.822+0000 7fab67e00640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 0x7fab4c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab58005fd0 tx=0x7fab58005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:16.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:16.825+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab500bd850 con 0x7fab68105b20 2024-04-26T21:30:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.062+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab681a3130 con 0x7fab68105b20 2024-04-26T21:30:17.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:16 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6836/847789147,v1:172.21.15.97:6837/847789147] up:boot 2024-04-26T21:30:17.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:16 smithi097 ceph-mon[28127]: fsmap 4 up:standby 2024-04-26T21:30:17.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:16 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:30:17.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:16 smithi097 ceph-mon[28127]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.063+0000 7fab65600640 1 -- 172.21.15.97:0/3278490762 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fab500bd150 con 0x7fab68105b20 2024-04-26T21:30:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:17.064 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:30:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 msgr2=0x7fab4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 0x7fab4c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab58005fd0 tx=0x7fab58005850 comp rx=0 tx=0).stop 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 msgr2=0x7fab6819b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab50005ab0 tx=0x7fab500042f0 comp rx=0 tx=0).stop 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 shutdown_connections 2024-04-26T21:30:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab68108450 0x7fab681a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fab4c0a3bb0 0x7fab4c0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab68105b20 0x7fab6819b050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 --2- 172.21.15.97:0/3278490762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab681031f0 0x7fab6819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 >> 172.21.15.97:0/3278490762 conn(0x7fab680fcbc0 msgr2=0x7fab68101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 shutdown_connections 2024-04-26T21:30:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:17.066+0000 7fab6d800640 1 -- 172.21.15.97:0/3278490762 wait complete. 2024-04-26T21:30:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:16 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6836/847789147,v1:172.21.15.97:6837/847789147] up:boot 2024-04-26T21:30:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:16 smithi111 ceph-mon[31001]: fsmap 4 up:standby 2024-04-26T21:30:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:16 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:30:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:16 smithi111 ceph-mon[31001]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:17.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:16 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6836/847789147,v1:172.21.15.97:6837/847789147] up:boot 2024-04-26T21:30:17.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:16 smithi143 ceph-mon[31461]: fsmap 4 up:standby 2024-04-26T21:30:17.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:16 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:30:17.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:16 smithi143 ceph-mon[31461]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:17 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3278490762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:18.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:17 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3278490762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:18.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:17 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3278490762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:18.623 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 systemd[1]: Starting Ceph mds.e for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:18 smithi111 ceph-mon[31001]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:18.893 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 podman[60619]: 2024-04-26 21:30:18.529128566 +0000 UTC m=+0.038560784 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:18.893 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 podman[60619]: 2024-04-26 21:30:18.643470043 +0000 UTC m=+0.152902283 container create 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T21:30:19.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:19.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:18 smithi097 ceph-mon[28127]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:18 smithi143 ceph-mon[31461]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 -- 172.21.15.97:0/816470703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa338105d10 msgr2=0x7fa338108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 --2- 172.21.15.97:0/816470703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa338105d10 0x7fa338108100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa320009900 tx=0x7fa32002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 -- 172.21.15.97:0/816470703 shutdown_connections 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 --2- 172.21.15.97:0/816470703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338108640 0x7fa33810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 --2- 172.21.15.97:0/816470703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa338105d10 0x7fa338108100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 --2- 172.21.15.97:0/816470703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3381033e0 0x7fa3381057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 -- 172.21.15.97:0/816470703 >> 172.21.15.97:0/816470703 conn(0x7fa3380fcdf0 msgr2=0x7fa3380ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 -- 172.21.15.97:0/816470703 shutdown_connections 2024-04-26T21:30:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.309+0000 7fa33ce00640 1 -- 172.21.15.97:0/816470703 wait complete. 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 Processor -- start 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 -- start start 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 0x7fa33819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa338108640 0x7fa3381a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810e3a0 con 0x7fa3381033e0 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810e220 con 0x7fa338105d10 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.310+0000 7fa33ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810e520 con 0x7fa338108640 2024-04-26T21:30:19.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50382/0 (socket says 172.21.15.97:50382) 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 -- 172.21.15.97:0/2200410045 learned_addr learned my addr 172.21.15.97:0/2200410045 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa336800640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa338108640 0x7fa3381a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa32fe00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 0x7fa33819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 -- 172.21.15.97:0/2200410045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa338108640 msgr2=0x7fa3381a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa338108640 0x7fa3381a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 -- 172.21.15.97:0/2200410045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 msgr2=0x7fa33819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 0x7fa33819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa335e00640 1 -- 172.21.15.97:0/2200410045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3381a2920 con 0x7fa3381033e0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.311+0000 7fa32fe00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 0x7fa33819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa335e00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e9f0 tx=0x7fa32800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32800cd80 con 0x7fa3381033e0 2024-04-26T21:30:19.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3280054b0 con 0x7fa3381033e0 2024-04-26T21:30:19.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328010430 con 0x7fa3381033e0 2024-04-26T21:30:19.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381a2c10 con 0x7fa3381033e0 2024-04-26T21:30:19.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.312+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a3110 con 0x7fa3381033e0 2024-04-26T21:30:19.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.313+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33810a190 con 0x7fa3381033e0 2024-04-26T21:30:19.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.315+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa3280046c0 con 0x7fa3381033e0 2024-04-26T21:30:19.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.317+0000 7fa32ea00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 0x7fa31c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:19.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.317+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa3280d04b0 con 0x7fa3381033e0 2024-04-26T21:30:19.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.317+0000 7fa32fe00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 0x7fa31c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:19.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.318+0000 7fa32fe00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 0x7fa31c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa32002a910 tx=0x7fa320000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:19.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.321+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa32809d290 con 0x7fa3381033e0 2024-04-26T21:30:19.392 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 podman[60619]: 2024-04-26 21:30:18.905540823 +0000 UTC m=+0.414973069 container init 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:30:19.393 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 podman[60619]: 2024-04-26 21:30:18.91590103 +0000 UTC m=+0.425333229 container start 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T21:30:19.393 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 ceph-mds[60638]: -- 172.21.15.111:0/390347830 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576017c7a40 con 0x55760179ac00 2024-04-26T21:30:19.393 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:18 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: starting mds.e at 2024-04-26T21:30:19.393 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:19 smithi111 bash[60619]: 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a 2024-04-26T21:30:19.393 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:30:19 smithi111 systemd[1]: Started Ceph mds.e for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.566+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3380635d0 con 0x7fa3381033e0 2024-04-26T21:30:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.567+0000 7fa32ea00640 1 -- 172.21.15.97:0/2200410045 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fa32809cb90 con 0x7fa3381033e0 2024-04-26T21:30:19.569 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:19.569 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:30:19.569 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-04-26T21:30:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 msgr2=0x7fa31c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 0x7fa31c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa32002a910 tx=0x7fa320000bc0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 msgr2=0x7fa33819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e9f0 tx=0x7fa32800edf0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 shutdown_connections 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa338108640 0x7fa3381a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa338105d10 0x7fa33819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa31c0a3bb0 0x7fa31c0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 --2- 172.21.15.97:0/2200410045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3381033e0 0x7fa33819ad10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 >> 172.21.15.97:0/2200410045 conn(0x7fa3380fcdf0 msgr2=0x7fa338102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 shutdown_connections 2024-04-26T21:30:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:19.570+0000 7fa33ce00640 1 -- 172.21.15.97:0/2200410045 wait complete. 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: Deploying daemon mds.f on smithi143 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2200410045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:boot 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: fsmap 5 up:standby 2024-04-26T21:30:20.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:20.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:20.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: Deploying daemon mds.f on smithi143 2024-04-26T21:30:20.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2200410045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:20.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:boot 2024-04-26T21:30:20.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: fsmap 5 up:standby 2024-04-26T21:30:20.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:30:20.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: Deploying daemon mds.f on smithi143 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2200410045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:boot 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: fsmap 5 up:standby 2024-04-26T21:30:20.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:30:21.455 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:21 smithi143 ceph-mon[31461]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:21.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:21 smithi097 ceph-mon[28127]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:21.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:21.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:21 smithi111 ceph-mon[31001]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 -- 172.21.15.97:0/1820621010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 msgr2=0x7f92b4079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 --2- 172.21.15.97:0/1820621010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b4079770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009ca0 tx=0x7f92ac02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 -- 172.21.15.97:0/1820621010 shutdown_connections 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 --2- 172.21.15.97:0/1820621010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b4079770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 --2- 172.21.15.97:0/1820621010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 0x7f92b4078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 --2- 172.21.15.97:0/1820621010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 -- 172.21.15.97:0/1820621010 >> 172.21.15.97:0/1820621010 conn(0x7f92b4100120 msgr2=0x7f92b4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 -- 172.21.15.97:0/1820621010 shutdown_connections 2024-04-26T21:30:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.810+0000 7f92bbc00640 1 -- 172.21.15.97:0/1820621010 wait complete. 2024-04-26T21:30:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 Processor -- start 2024-04-26T21:30:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 -- start start 2024-04-26T21:30:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 0x7f92b419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41149e0 con 0x7f92b4078ed0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4114860 con 0x7f92b40772f0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.811+0000 7f92bbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4114b60 con 0x7f92b4078530 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35916/0 (socket says 172.21.15.97:35916) 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 -- 172.21.15.97:0/2266988789 learned_addr learned my addr 172.21.15.97:0/2266988789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b3e00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 0x7f92b419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9200640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 -- 172.21.15.97:0/2266988789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 msgr2=0x7f92b419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 0x7f92b419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 -- 172.21.15.97:0/2266988789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 msgr2=0x7f92b419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9c00640 1 -- 172.21.15.97:0/2266988789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41a6d10 con 0x7f92b4078ed0 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.812+0000 7f92b9200640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92b9c00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92ac031040 tx=0x7f92ac003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004040 con 0x7f92b4078ed0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92ac004670 con 0x7f92b4078ed0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac030c60 con 0x7f92b4078ed0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41a6fa0 con 0x7f92b4078ed0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.813+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41a7410 con 0x7f92b4078ed0 2024-04-26T21:30:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.814+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b41114e0 con 0x7f92b4078ed0 2024-04-26T21:30:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.816+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f92ac0041e0 con 0x7f92b4078ed0 2024-04-26T21:30:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.818+0000 7f92b1600640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 0x7f929c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.818+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f92ac0f1770 con 0x7f92b4078ed0 2024-04-26T21:30:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.818+0000 7f92b9200640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 0x7f929c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.818+0000 7f92b9200640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 0x7f929c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009b40 tx=0x7f92a8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:21.822+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f92ac03a390 con 0x7f92b4078ed0 2024-04-26T21:30:22.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.068+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b40635d0 con 0x7f92b4078ed0 2024-04-26T21:30:22.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.070+0000 7f92b1600640 1 -- 172.21.15.97:0/2266988789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f92ac0be550 con 0x7f92b4078ed0 2024-04-26T21:30:22.071 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:22.071 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:30:22.071 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-04-26T21:30:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.072+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 msgr2=0x7f929c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.072+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 0x7f929c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009b40 tx=0x7f92a8005e50 comp rx=0 tx=0).stop 2024-04-26T21:30:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.072+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 msgr2=0x7f92b41a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.072+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92ac031040 tx=0x7f92ac003ba0 comp rx=0 tx=0).stop 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 shutdown_connections 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f929c0a3bb0 0x7f929c0a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4078ed0 0x7f92b41a66a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92b4078530 0x7f92b419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 --2- 172.21.15.97:0/2266988789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92b40772f0 0x7f92b419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 >> 172.21.15.97:0/2266988789 conn(0x7f92b4100120 msgr2=0x7f92b4101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 shutdown_connections 2024-04-26T21:30:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:22.073+0000 7f92bbc00640 1 -- 172.21.15.97:0/2266988789 wait complete. 2024-04-26T21:30:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:22 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2266988789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:22.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2266988789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:22.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:22 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2266988789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:22.686 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:22 smithi143 systemd[1]: Starting Ceph mds.f for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:22.996 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:22 smithi143 podman[61466]: 2024-04-26 21:30:22.591903926 +0000 UTC m=+0.035668354 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:22.996 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:22 smithi143 podman[61466]: 2024-04-26 21:30:22.746216959 +0000 UTC m=+0.189981443 container create cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:30:22.996 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:22 smithi143 podman[61466]: 2024-04-26 21:30:22.994166588 +0000 UTC m=+0.437931049 container init cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:23 smithi143 ceph-mon[31461]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:23 smithi143 podman[61466]: 2024-04-26 21:30:23.006411117 +0000 UTC m=+0.450175544 container start cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:23 smithi143 ceph-mds[61484]: -- 172.21.15.143:0/3387259122 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555877f91a40 con 0x555877f65000 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:23 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: starting mds.f at 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:23 smithi143 bash[61466]: cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 2024-04-26T21:30:23.295 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:30:23 smithi143 systemd[1]: Started Ceph mds.f for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:23.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:23 smithi097 ceph-mon[28127]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:23.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:23 smithi111 ceph-mon[31001]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:24.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.304+0000 7f2066800640 1 -- 172.21.15.97:0/186164025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 msgr2=0x7f20601111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.304+0000 7f2066800640 1 --2- 172.21.15.97:0/186164025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601111d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2058009900 tx=0x7f205802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 -- 172.21.15.97:0/186164025 shutdown_connections 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 --2- 172.21.15.97:0/186164025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601111d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 --2- 172.21.15.97:0/186164025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060109f80 0x7f206010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 --2- 172.21.15.97:0/186164025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060108d90 0x7f20601091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 -- 172.21.15.97:0/186164025 >> 172.21.15.97:0/186164025 conn(0x7f2060077d30 msgr2=0x7f206007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 -- 172.21.15.97:0/186164025 shutdown_connections 2024-04-26T21:30:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.305+0000 7f2066800640 1 -- 172.21.15.97:0/186164025 wait complete. 2024-04-26T21:30:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 Processor -- start 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 -- start start 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060108d90 0x7f206019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 0x7f206019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2060114770 con 0x7f206010a940 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20601145f0 con 0x7f2060108d90 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2066800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20601148f0 con 0x7f2060109f80 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2065e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2065e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35940/0 (socket says 172.21.15.97:35940) 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.306+0000 7f2065e00640 1 -- 172.21.15.97:0/3869791230 learned_addr learned my addr 172.21.15.97:0/3869791230 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065400640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060108d90 0x7f206019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2064a00640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 0x7f206019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 -- 172.21.15.97:0/3869791230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 msgr2=0x7f206019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:24.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 0x7f206019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 -- 172.21.15.97:0/3869791230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060108d90 msgr2=0x7f206019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060108d90 0x7f206019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 -- 172.21.15.97:0/3869791230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20601a6a70 con 0x7f206010a940 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2064a00640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 0x7f206019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2065e00640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2058004770 tx=0x7f20580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205803f030 con 0x7f206010a940 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20601a6d00 con 0x7f206010a940 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2058003c90 con 0x7f206010a940 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.307+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058035070 con 0x7f206010a940 2024-04-26T21:30:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.308+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20601a71a0 con 0x7f206010a940 2024-04-26T21:30:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.309+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2060110220 con 0x7f206010a940 2024-04-26T21:30:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.311+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2058042030 con 0x7f206010a940 2024-04-26T21:30:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.312+0000 7f205e000640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 0x7f20480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.312+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f20580f0780 con 0x7f206010a940 2024-04-26T21:30:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.312+0000 7f2065400640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 0x7f20480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.313+0000 7f2065400640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 0x7f20480a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2060109de0 tx=0x7f2054009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.316+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f20580bd560 con 0x7f206010a940 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6834/4086222297,v1:172.21.15.143:6835/4086222297] up:boot 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: fsmap 6 up:standby 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:24.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:24 smithi111 ceph-mon[31001]: Deploying daemon mds.c on smithi143 2024-04-26T21:30:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6834/4086222297,v1:172.21.15.143:6835/4086222297] up:boot 2024-04-26T21:30:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: fsmap 6 up:standby 2024-04-26T21:30:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:30:24.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:24.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:24 smithi143 ceph-mon[31461]: Deploying daemon mds.c on smithi143 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6834/4086222297,v1:172.21.15.143:6835/4086222297] up:boot 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: fsmap 6 up:standby 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:24.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:24 smithi097 ceph-mon[28127]: Deploying daemon mds.c on smithi143 2024-04-26T21:30:24.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.563+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20600635d0 con 0x7f206010a940 2024-04-26T21:30:24.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.564+0000 7f205e000640 1 -- 172.21.15.97:0/3869791230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f20580bce60 con 0x7f206010a940 2024-04-26T21:30:24.565 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:24.565 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:30:24.566 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 msgr2=0x7f20480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 0x7f20480a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2060109de0 tx=0x7f2054009870 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 msgr2=0x7f20601a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2058004770 tx=0x7f20580042f0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 shutdown_connections 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f20480a3bb0 0x7f20480a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206010a940 0x7f20601a6400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2060109f80 0x7f206019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 --2- 172.21.15.97:0/3869791230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2060108d90 0x7f206019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 >> 172.21.15.97:0/3869791230 conn(0x7f2060077d30 msgr2=0x7f206010d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:24.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 shutdown_connections 2024-04-26T21:30:24.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:24.567+0000 7f2066800640 1 -- 172.21.15.97:0/3869791230 wait complete. 2024-04-26T21:30:25.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:25 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3869791230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:25.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:25 smithi143 ceph-mon[31461]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:25.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:25 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3869791230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:25.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:25 smithi097 ceph-mon[28127]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:25.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:25 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3869791230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:25.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:25 smithi111 ceph-mon[31001]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:26.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:26.795 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:26 smithi143 systemd[1]: Starting Ceph mds.c for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:26.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 -- 172.21.15.97:0/1798404557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 msgr2=0x7f8f541091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 --2- 172.21.15.97:0/1798404557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f541091b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009900 tx=0x7f8f4802a410 comp rx=0 tx=0).stop 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 -- 172.21.15.97:0/1798404557 shutdown_connections 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 --2- 172.21.15.97:0/1798404557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f5410a940 0x7f8f541111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 --2- 172.21.15.97:0/1798404557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f54109f80 0x7f8f5410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 --2- 172.21.15.97:0/1798404557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f541091b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 -- 172.21.15.97:0/1798404557 >> 172.21.15.97:0/1798404557 conn(0x7f8f54077d30 msgr2=0x7f8f5407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.799+0000 7f8f5b200640 1 -- 172.21.15.97:0/1798404557 shutdown_connections 2024-04-26T21:30:26.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.800+0000 7f8f5b200640 1 -- 172.21.15.97:0/1798404557 wait complete. 2024-04-26T21:30:26.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.800+0000 7f8f5b200640 1 Processor -- start 2024-04-26T21:30:26.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 -- start start 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54109f80 0x7f8f5419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 0x7f8f541a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54114740 con 0x7f8f54108d90 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541145c0 con 0x7f8f54109f80 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f5b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541148c0 con 0x7f8f5410a940 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35962/0 (socket says 172.21.15.97:35962) 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.801+0000 7f8f59e00640 1 -- 172.21.15.97:0/2567035526 learned_addr learned my addr 172.21.15.97:0/2567035526 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f5a800640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 0x7f8f541a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59400640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54109f80 0x7f8f5419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 -- 172.21.15.97:0/2567035526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 msgr2=0x7f8f541a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:26.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 0x7f8f541a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 -- 172.21.15.97:0/2567035526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54109f80 msgr2=0x7f8f5419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54109f80 0x7f8f5419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 -- 172.21.15.97:0/2567035526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f541a6b00 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f5a800640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 0x7f8f541a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f59e00640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005ab0 tx=0x7f8f480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4803f030 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f48003c90 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48035070 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.802+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f541a6d90 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.803+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541a71d0 con 0x7f8f54108d90 2024-04-26T21:30:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.804+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54110270 con 0x7f8f54108d90 2024-04-26T21:30:26.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.806+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f48042030 con 0x7f8f54108d90 2024-04-26T21:30:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.807+0000 7f8f4ea00640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 0x7f8f3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.807+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8f480f0a70 con 0x7f8f54108d90 2024-04-26T21:30:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.807+0000 7f8f59400640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 0x7f8f3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.808+0000 7f8f59400640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 0x7f8f3c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:26.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:26.812+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f480bd850 con 0x7f8f54108d90 2024-04-26T21:30:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.054+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f5402dcf0 con 0x7f8f54108d90 2024-04-26T21:30:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.055+0000 7f8f4ea00640 1 -- 172.21.15.97:0/2567035526 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f8f480bd150 con 0x7f8f54108d90 2024-04-26T21:30:27.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:27.057 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:30:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 msgr2=0x7f8f3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 0x7f8f3c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005850 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 msgr2=0x7f8f5419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005ab0 tx=0x7f8f480042f0 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 shutdown_connections 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f5410a940 0x7f8f541a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54109f80 0x7f8f5419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8f3c0a3bb0 0x7f8f3c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 --2- 172.21.15.97:0/2567035526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f54108d90 0x7f8f5419ef30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 >> 172.21.15.97:0/2567035526 conn(0x7f8f54077d30 msgr2=0x7f8f5410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:27.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 shutdown_connections 2024-04-26T21:30:27.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:27.058+0000 7f8f5b200640 1 -- 172.21.15.97:0/2567035526 wait complete. 2024-04-26T21:30:27.134 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:26 smithi143 podman[61764]: 2024-04-26 21:30:26.731642195 +0000 UTC m=+0.037757916 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:27.134 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:26 smithi143 podman[61764]: 2024-04-26 21:30:26.83646213 +0000 UTC m=+0.142577872 container create e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T21:30:27.134 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 podman[61764]: 2024-04-26 21:30:27.132137208 +0000 UTC m=+0.438253013 container init e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T21:30:27.545 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 podman[61764]: 2024-04-26 21:30:27.142880737 +0000 UTC m=+0.448996456 container start e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:30:27.545 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mds[61783]: -- 172.21.15.143:0/3497869642 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56452aa37a40 con 0x56452aa0b000 2024-04-26T21:30:27.546 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: starting mds.c at 2024-04-26T21:30:27.546 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 bash[61764]: e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 2024-04-26T21:30:27.546 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:30:27 smithi143 systemd[1]: Started Ceph mds.c for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2567035526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:28.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:28.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2567035526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:28.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:28.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2567035526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:28 smithi143 ceph-mon[31461]: Deploying daemon mds.h on smithi111 2024-04-26T21:30:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:28 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6836/1658655529,v1:172.21.15.143:6837/1658655529] up:boot 2024-04-26T21:30:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:28 smithi143 ceph-mon[31461]: fsmap 7 up:standby 2024-04-26T21:30:29.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:30:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:28 smithi097 ceph-mon[28127]: Deploying daemon mds.h on smithi111 2024-04-26T21:30:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:28 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6836/1658655529,v1:172.21.15.143:6837/1658655529] up:boot 2024-04-26T21:30:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:28 smithi097 ceph-mon[28127]: fsmap 7 up:standby 2024-04-26T21:30:29.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:30:29.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:29.100 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:28 smithi111 ceph-mon[31001]: Deploying daemon mds.h on smithi111 2024-04-26T21:30:29.101 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:28 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6836/1658655529,v1:172.21.15.143:6837/1658655529] up:boot 2024-04-26T21:30:29.101 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:28 smithi111 ceph-mon[31001]: fsmap 7 up:standby 2024-04-26T21:30:29.101 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.289+0000 7fbe49600640 1 -- 172.21.15.97:0/3273933210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44104320 msgr2=0x7fbe4410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.289+0000 7fbe49600640 1 --2- 172.21.15.97:0/3273933210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44104320 0x7fbe4410abb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009ca0 tx=0x7fbe3c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.289+0000 7fbe49600640 1 -- 172.21.15.97:0/3273933210 shutdown_connections 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.289+0000 7fbe49600640 1 --2- 172.21.15.97:0/3273933210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44104320 0x7fbe4410abb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 --2- 172.21.15.97:0/3273933210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44103960 0x7fbe44103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 --2- 172.21.15.97:0/3273933210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44102770 0x7fbe44102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 -- 172.21.15.97:0/3273933210 >> 172.21.15.97:0/3273933210 conn(0x7fbe440fdcc0 msgr2=0x7fbe44100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 -- 172.21.15.97:0/3273933210 shutdown_connections 2024-04-26T21:30:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 -- 172.21.15.97:0/3273933210 wait complete. 2024-04-26T21:30:29.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.290+0000 7fbe49600640 1 Processor -- start 2024-04-26T21:30:29.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 -- start start 2024-04-26T21:30:29.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 0x7fbe4419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 0x7fbe441a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe44075d70 con 0x7fbe44103960 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe44075bf0 con 0x7fbe44102770 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe49600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe44075ef0 con 0x7fbe44104320 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe43400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe43400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35978/0 (socket says 172.21.15.97:35978) 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.291+0000 7fbe43400640 1 -- 172.21.15.97:0/3694216089 learned_addr learned my addr 172.21.15.97:0/3694216089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43e00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 0x7fbe4419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 -- 172.21.15.97:0/3694216089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 msgr2=0x7fbe441a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe48c00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 0x7fbe441a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:29.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 0x7fbe441a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 -- 172.21.15.97:0/3694216089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 msgr2=0x7fbe4419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 0x7fbe4419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 -- 172.21.15.97:0/3694216089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe441a26a0 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43e00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 0x7fbe4419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe48c00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 0x7fbe441a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe43400640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c890 tx=0x7fbe3000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004220 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe30007e40 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe441a2990 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004a00 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.292+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441a2ee0 con 0x7fbe44103960 2024-04-26T21:30:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.293+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe441092b0 con 0x7fbe44103960 2024-04-26T21:30:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.296+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbe300079b0 con 0x7fbe44103960 2024-04-26T21:30:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.298+0000 7fbe40c00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 0x7fbe280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:29.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.298+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fbe300cfd80 con 0x7fbe44103960 2024-04-26T21:30:29.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.298+0000 7fbe43e00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 0x7fbe280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:29.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.299+0000 7fbe43e00640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 0x7fbe280a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:29.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.302+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe30014030 con 0x7fbe44103960 2024-04-26T21:30:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.539+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe4402dad0 con 0x7fbe44103960 2024-04-26T21:30:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.539+0000 7fbe40c00640 1 -- 172.21.15.97:0/3694216089 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fbe3009cb60 con 0x7fbe44103960 2024-04-26T21:30:29.541 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:29.542 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:30:29.542 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 msgr2=0x7fbe280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 0x7fbe280a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34008040 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 msgr2=0x7fbe4419afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c890 tx=0x7fbe3000cd50 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 shutdown_connections 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe44104320 0x7fbe441a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fbe280a3bb0 0x7fbe280a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe44103960 0x7fbe4419afc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 --2- 172.21.15.97:0/3694216089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe44102770 0x7fbe4419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 >> 172.21.15.97:0/3694216089 conn(0x7fbe440fdcc0 msgr2=0x7fbe440ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 shutdown_connections 2024-04-26T21:30:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:29.542+0000 7fbe49600640 1 -- 172.21.15.97:0/3694216089 wait complete. 2024-04-26T21:30:29.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:29 smithi097 ceph-mon[28127]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:29.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:29 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3694216089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:29.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:29 smithi111 ceph-mon[31001]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:29.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:29 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3694216089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:29 smithi143 ceph-mon[31461]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:29 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3694216089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:30.775 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:30 smithi111 systemd[1]: Starting Ceph mds.h for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:31.140 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:30 smithi111 podman[60921]: 2024-04-26 21:30:30.678177435 +0000 UTC m=+0.033016262 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:31.140 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:30 smithi111 podman[60921]: 2024-04-26 21:30:30.800127529 +0000 UTC m=+0.154966371 container create 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:30:31.140 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 podman[60921]: 2024-04-26 21:30:31.137636436 +0000 UTC m=+0.492475284 container init 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:30:31.392 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 podman[60921]: 2024-04-26 21:30:31.148049846 +0000 UTC m=+0.502888695 container start 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T21:30:31.393 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mds[60939]: -- 172.21.15.111:0/1173038714 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e636a8ba40 con 0x55e636a5e800 2024-04-26T21:30:31.393 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: starting mds.h at 2024-04-26T21:30:31.393 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 bash[60921]: 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 2024-04-26T21:30:31.393 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:30:31 smithi111 systemd[1]: Started Ceph mds.h for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:31.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:31.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 -- 172.21.15.97:0/2435272814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c104300 msgr2=0x7fd43c10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 --2- 172.21.15.97:0/2435272814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c104300 0x7fd43c10ab90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd434009ca0 tx=0x7fd43402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 -- 172.21.15.97:0/2435272814 shutdown_connections 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 --2- 172.21.15.97:0/2435272814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c104300 0x7fd43c10ab90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 --2- 172.21.15.97:0/2435272814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c103940 0x7fd43c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 --2- 172.21.15.97:0/2435272814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c102750 0x7fd43c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 -- 172.21.15.97:0/2435272814 >> 172.21.15.97:0/2435272814 conn(0x7fd43c0fdcc0 msgr2=0x7fd43c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 -- 172.21.15.97:0/2435272814 shutdown_connections 2024-04-26T21:30:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.767+0000 7fd442400640 1 -- 172.21.15.97:0/2435272814 wait complete. 2024-04-26T21:30:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.768+0000 7fd442400640 1 Processor -- start 2024-04-26T21:30:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.768+0000 7fd442400640 1 -- start start 2024-04-26T21:30:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.768+0000 7fd442400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c102750 0x7fd43c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd442400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd442400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c104300 0x7fd43c1a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd442400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c10e180 con 0x7fd43c103940 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd442400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c10e000 con 0x7fd43c102750 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd442400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c10e300 con 0x7fd43c104300 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33666/0 (socket says 172.21.15.97:33666) 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 -- 172.21.15.97:0/2026401403 learned_addr learned my addr 172.21.15.97:0/2026401403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd441000640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c102750 0x7fd43c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd441a00640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c104300 0x7fd43c1a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 -- 172.21.15.97:0/2026401403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c104300 msgr2=0x7fd43c1a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c104300 0x7fd43c1a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 -- 172.21.15.97:0/2026401403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c102750 msgr2=0x7fd43c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c102750 0x7fd43c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.769+0000 7fd433e00640 1 -- 172.21.15.97:0/2026401403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c1a2660 con 0x7fd43c103940 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd433e00640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d950 tx=0x7fd42400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424017070 con 0x7fd43c103940 2024-04-26T21:30:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd424009c30 con 0x7fd43c103940 2024-04-26T21:30:31.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c1a2950 con 0x7fd43c103940 2024-04-26T21:30:31.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400f620 con 0x7fd43c103940 2024-04-26T21:30:31.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.770+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1a2ea0 con 0x7fd43c103940 2024-04-26T21:30:31.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.771+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c1a30d0 con 0x7fd43c103940 2024-04-26T21:30:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.773+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd424010460 con 0x7fd43c103940 2024-04-26T21:30:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.775+0000 7fd43a000640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 0x7fd4200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.775+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd4240cfcc0 con 0x7fd43c103940 2024-04-26T21:30:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.775+0000 7fd441000640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 0x7fd4200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.776+0000 7fd441000640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 0x7fd4200a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:31.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:31.779+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd4240d6080 con 0x7fd43c103940 2024-04-26T21:30:32.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.020+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43c1a32a0 con 0x7fd43c103940 2024-04-26T21:30:32.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.021+0000 7fd43a000640 1 -- 172.21.15.97:0/2026401403 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7fd42409caa0 con 0x7fd43c103940 2024-04-26T21:30:32.022 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:32.023 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:32.023 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-26T21:30:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 msgr2=0x7fd4200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 0x7fd4200a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).stop 2024-04-26T21:30:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 msgr2=0x7fd43c19af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d950 tx=0x7fd42400de10 comp rx=0 tx=0).stop 2024-04-26T21:30:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 shutdown_connections 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd43c104300 0x7fd43c1a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4200a3bb0 0x7fd4200a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd43c103940 0x7fd43c19af80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 --2- 172.21.15.97:0/2026401403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd43c102750 0x7fd43c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 >> 172.21.15.97:0/2026401403 conn(0x7fd43c0fdcc0 msgr2=0x7fd43c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 shutdown_connections 2024-04-26T21:30:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:32.024+0000 7fd442400640 1 -- 172.21.15.97:0/2026401403 wait complete. 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:32.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:32.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:32 smithi143 ceph-mon[31461]: Deploying daemon mds.i on smithi143 2024-04-26T21:30:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:32 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:boot 2024-04-26T21:30:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:32 smithi143 ceph-mon[31461]: fsmap 8 up:standby 2024-04-26T21:30:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:30:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:32 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2026401403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:32 smithi097 ceph-mon[28127]: Deploying daemon mds.i on smithi143 2024-04-26T21:30:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:32 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:boot 2024-04-26T21:30:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:32 smithi097 ceph-mon[28127]: fsmap 8 up:standby 2024-04-26T21:30:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:30:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:32 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2026401403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:32 smithi111 ceph-mon[31001]: Deploying daemon mds.i on smithi143 2024-04-26T21:30:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:32 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:boot 2024-04-26T21:30:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:32 smithi111 ceph-mon[31001]: fsmap 8 up:standby 2024-04-26T21:30:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:30:33.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:32 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2026401403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:33 smithi143 ceph-mon[31461]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:34.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:34.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:33 smithi097 ceph-mon[28127]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:33 smithi111 ceph-mon[31001]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 -- 172.21.15.97:0/3318011065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5390108f90 msgr2=0x7f53901093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 --2- 172.21.15.97:0/3318011065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5390108f90 0x7f53901093b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5380009900 tx=0x7f538002a400 comp rx=0 tx=0).stop 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 -- 172.21.15.97:0/3318011065 shutdown_connections 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 --2- 172.21.15.97:0/3318011065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f539010ab40 0x7f53901113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 --2- 172.21.15.97:0/3318011065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010a180 0x7f539010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 --2- 172.21.15.97:0/3318011065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5390108f90 0x7f53901093b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 -- 172.21.15.97:0/3318011065 >> 172.21.15.97:0/3318011065 conn(0x7f5390077f40 msgr2=0x7f539007a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 -- 172.21.15.97:0/3318011065 shutdown_connections 2024-04-26T21:30:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.260+0000 7f5395200640 1 -- 172.21.15.97:0/3318011065 wait complete. 2024-04-26T21:30:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.261+0000 7f5395200640 1 Processor -- start 2024-04-26T21:30:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.261+0000 7f5395200640 1 -- start start 2024-04-26T21:30:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 0x7f539019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010ab40 0x7f53901a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5390114990 con 0x7f539010a180 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5390114810 con 0x7f5390108f90 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5395200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5390114b10 con 0x7f539010ab40 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33680/0 (socket says 172.21.15.97:33680) 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 -- 172.21.15.97:0/1704219511 learned_addr learned my addr 172.21.15.97:0/1704219511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f538e800640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010ab40 0x7f53901a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f538de00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 0x7f539019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 -- 172.21.15.97:0/1704219511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010ab40 msgr2=0x7f53901a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010ab40 0x7f53901a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 -- 172.21.15.97:0/1704219511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 msgr2=0x7f539019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 0x7f539019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f5387e00640 1 -- 172.21.15.97:0/1704219511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a6c80 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.262+0000 7f538de00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 0x7f539019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5387e00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f537800d950 tx=0x7f537800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378017030 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5378004020 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a6f70 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780055d0 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.263+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a7430 con 0x7f539010a180 2024-04-26T21:30:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.264+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53901102b0 con 0x7f539010a180 2024-04-26T21:30:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.266+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5378009a30 con 0x7f539010a180 2024-04-26T21:30:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.268+0000 7f5386a00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 0x7f53740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.268+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f53780cf550 con 0x7f539010a180 2024-04-26T21:30:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.268+0000 7f538de00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 0x7f53740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:34.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.269+0000 7f538de00640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 0x7f53740a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f538002a920 tx=0x7f5380000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:34.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.272+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f537809c330 con 0x7f539010a180 2024-04-26T21:30:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.511+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f539002dad0 con 0x7f539010a180 2024-04-26T21:30:34.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.512+0000 7f5386a00640 1 -- 172.21.15.97:0/1704219511 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f537809bc30 con 0x7f539010a180 2024-04-26T21:30:34.513 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:34.513 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 msgr2=0x7f53740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 0x7f53740a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f538002a920 tx=0x7f5380000bc0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 msgr2=0x7f539019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f537800d950 tx=0x7f537800de10 comp rx=0 tx=0).stop 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 shutdown_connections 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539010ab40 0x7f53901a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f53740a3bb0 0x7f53740a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f539010a180 0x7f539019f5a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 --2- 172.21.15.97:0/1704219511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5390108f90 0x7f539019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 >> 172.21.15.97:0/1704219511 conn(0x7f5390077f40 msgr2=0x7f539010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 shutdown_connections 2024-04-26T21:30:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:34.515+0000 7f5395200640 1 -- 172.21.15.97:0/1704219511 wait complete. 2024-04-26T21:30:34.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:34 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1704219511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:34.902 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:34 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1704219511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:35.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:34 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1704219511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:35.280 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 podman[62072]: 2024-04-26 21:30:35.007862379 +0000 UTC m=+0.160333164 container create a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T21:30:35.280 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 podman[62072]: 2024-04-26 21:30:35.278469249 +0000 UTC m=+0.430940074 container init a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T21:30:35.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 podman[62072]: 2024-04-26 21:30:35.288914996 +0000 UTC m=+0.441385753 container start a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:30:35.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mds[62098]: -- 172.21.15.143:0/3294537760 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d087aa4e0 con 0x558d08780c00 2024-04-26T21:30:35.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mds[62098]: -- 172.21.15.143:0/3294537760 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d087ada40 con 0x558d08780c00 2024-04-26T21:30:35.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: starting mds.i at 2024-04-26T21:30:35.546 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 bash[62072]: a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 2024-04-26T21:30:35.546 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:30:35 smithi143 systemd[1]: Started Ceph mds.i for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:36.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:36.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 -- 172.21.15.97:0/4084486107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 msgr2=0x7f28f010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 --2- 172.21.15.97:0/4084486107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f010a400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 -- 172.21.15.97:0/4084486107 shutdown_connections 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 --2- 172.21.15.97:0/4084486107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 0x7f28f01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 --2- 172.21.15.97:0/4084486107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f010a400 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 --2- 172.21.15.97:0/4084486107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 0x7f28f01091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.731+0000 7f28ef200640 1 -- 172.21.15.97:0/4084486107 >> 172.21.15.97:0/4084486107 conn(0x7f28f0077d30 msgr2=0x7f28f007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.732+0000 7f28ef200640 1 -- 172.21.15.97:0/4084486107 shutdown_connections 2024-04-26T21:30:36.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.732+0000 7f28ef200640 1 -- 172.21.15.97:0/4084486107 wait complete. 2024-04-26T21:30:36.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.732+0000 7f28ef200640 1 Processor -- start 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 -- start start 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 0x7f28f019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 0x7f28f01a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01147c0 con 0x7f28f0109f80 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0114640 con 0x7f28f0108d90 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ef200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0114940 con 0x7f28f010a940 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33710/0 (socket says 172.21.15.97:33710) 2024-04-26T21:30:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ed400640 1 -- 172.21.15.97:0/1222376515 learned_addr learned my addr 172.21.15.97:0/1222376515 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.733+0000 7f28ede00640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 0x7f28f019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ee800640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 0x7f28f01a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ed400640 1 -- 172.21.15.97:0/1222376515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 msgr2=0x7f28f01a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ed400640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 0x7f28f01a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ed400640 1 -- 172.21.15.97:0/1222376515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 msgr2=0x7f28f019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ed400640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 0x7f28f019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.734+0000 7f28ed400640 1 -- 172.21.15.97:0/1222376515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01a6ab0 con 0x7f28f0109f80 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28ed400640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031460 tx=0x7f28dc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc03f070 con 0x7f28f0109f80 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28dc031da0 con 0x7f28f0109f80 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a6d40 con 0x7f28f0109f80 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc035030 con 0x7f28f0109f80 2024-04-26T21:30:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.735+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a71d0 con 0x7f28f0109f80 2024-04-26T21:30:36.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.736+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f0110180 con 0x7f28f0109f80 2024-04-26T21:30:36.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.738+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f28dc042070 con 0x7f28f0109f80 2024-04-26T21:30:36.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.740+0000 7f28e6a00640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 0x7f28d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:36.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.740+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f28dc0f0920 con 0x7f28f0109f80 2024-04-26T21:30:36.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.740+0000 7f28ede00640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 0x7f28d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:36.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.741+0000 7f28ede00640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 0x7f28d40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005db0 tx=0x7f28e0005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:36.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.744+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f28dc0bd700 con 0x7f28f0109f80 2024-04-26T21:30:36.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:36 smithi097 ceph-mon[28127]: Deploying daemon mds.g on smithi097 2024-04-26T21:30:36.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:36 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:boot 2024-04-26T21:30:36.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:36 smithi097 ceph-mon[28127]: fsmap 9 up:standby 2024-04-26T21:30:36.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:30:36.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.980+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28f00635d0 con 0x7f28f0109f80 2024-04-26T21:30:36.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.980+0000 7f28e6a00640 1 -- 172.21.15.97:0/1222376515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f28dc0bd000 con 0x7f28f0109f80 2024-04-26T21:30:36.981 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:36.981 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:36.982 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.982+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 msgr2=0x7f28d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.982+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 0x7f28d40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005db0 tx=0x7f28e0005d40 comp rx=0 tx=0).stop 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 msgr2=0x7f28f019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031460 tx=0x7f28dc003c80 comp rx=0 tx=0).stop 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 shutdown_connections 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f010a940 0x7f28f01a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f28d40a3bb0 0x7f28d40a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0109f80 0x7f28f019f3d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 --2- 172.21.15.97:0/1222376515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28f0108d90 0x7f28f019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 >> 172.21.15.97:0/1222376515 conn(0x7f28f0077d30 msgr2=0x7f28f010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 shutdown_connections 2024-04-26T21:30:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:36.983+0000 7f28ef200640 1 -- 172.21.15.97:0/1222376515 wait complete. 2024-04-26T21:30:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:36 smithi143 ceph-mon[31461]: Deploying daemon mds.g on smithi097 2024-04-26T21:30:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:36 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:boot 2024-04-26T21:30:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:36 smithi143 ceph-mon[31461]: fsmap 9 up:standby 2024-04-26T21:30:37.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:30:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:36 smithi111 ceph-mon[31001]: Deploying daemon mds.g on smithi097 2024-04-26T21:30:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:36 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:boot 2024-04-26T21:30:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:36 smithi111 ceph-mon[31001]: fsmap 9 up:standby 2024-04-26T21:30:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:30:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:37 smithi143 ceph-mon[31461]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:37 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1222376515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:37 smithi097 ceph-mon[28127]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:38.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:37 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1222376515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:37 smithi111 ceph-mon[31001]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:37 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1222376515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:39.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:39.063 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:38 smithi097 podman[69413]: 2024-04-26 21:30:38.848990004 +0000 UTC m=+0.152368262 container create 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:30:39.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.188+0000 7fdad3200640 1 -- 172.21.15.97:0/3522231932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 msgr2=0x7fdacc078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.188+0000 7fdad3200640 1 --2- 172.21.15.97:0/3522231932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc078990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009900 tx=0x7fdab802a410 comp rx=0 tx=0).stop 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 -- 172.21.15.97:0/3522231932 shutdown_connections 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 --2- 172.21.15.97:0/3522231932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc078ed0 0x7fdacc079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 --2- 172.21.15.97:0/3522231932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc078990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 --2- 172.21.15.97:0/3522231932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc0772f0 0x7fdacc077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 -- 172.21.15.97:0/3522231932 >> 172.21.15.97:0/3522231932 conn(0x7fdacc100120 msgr2=0x7fdacc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 -- 172.21.15.97:0/3522231932 shutdown_connections 2024-04-26T21:30:39.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 -- 172.21.15.97:0/3522231932 wait complete. 2024-04-26T21:30:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.189+0000 7fdad3200640 1 Processor -- start 2024-04-26T21:30:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 -- start start 2024-04-26T21:30:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 0x7fdacc19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc078ed0 0x7fdacc1a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114980 con 0x7fdacc078530 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114800 con 0x7fdacc078ed0 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114b00 con 0x7fdacc0772f0 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdacb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdacb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33746/0 (socket says 172.21.15.97:33746) 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdacb400640 1 -- 172.21.15.97:0/3595810141 learned_addr learned my addr 172.21.15.97:0/3595810141 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdacbe00640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 0x7fdacc19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.190+0000 7fdad1200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc078ed0 0x7fdacc1a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 -- 172.21.15.97:0/3595810141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 msgr2=0x7fdacc19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 0x7fdacc19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 -- 172.21.15.97:0/3595810141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc078ed0 msgr2=0x7fdacc1a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc078ed0 0x7fdacc1a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 -- 172.21.15.97:0/3595810141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc1a6cd0 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacbe00640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 0x7fdacc19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdacb400640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009230 tx=0x7fdab80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab803f030 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdab8003c90 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc1a6f60 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8035070 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.191+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a73c0 con 0x7fdacc078530 2024-04-26T21:30:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.193+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc111780 con 0x7fdacc078530 2024-04-26T21:30:39.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.195+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdab8042030 con 0x7fdacc078530 2024-04-26T21:30:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.196+0000 7fdac8c00640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.196+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdab80f0c00 con 0x7fdacc078530 2024-04-26T21:30:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.196+0000 7fdacbe00640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.197+0000 7fdacbe00640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 0x7fdab00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009b40 tx=0x7fdabc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:39.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.198+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdab80bd9e0 con 0x7fdacc078530 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 podman[69413]: 2024-04-26 21:30:39.120470018 +0000 UTC m=+0.423848247 container init 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 podman[69413]: 2024-04-26 21:30:39.127979707 +0000 UTC m=+0.431357929 container start 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mds[69459]: -- 172.21.15.97:0/1486704989 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5caf77a40 con 0x55d5caf4a800 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: starting mds.g at 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 bash[69413]: 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b 2024-04-26T21:30:39.314 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:30:39 smithi097 systemd[1]: Started Ceph mds.g for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:39.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.445+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdacc02dad0 con 0x7fdacc078530 2024-04-26T21:30:39.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.446+0000 7fdac8c00640 1 -- 172.21.15.97:0/3595810141 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fdab80bd2e0 con 0x7fdacc078530 2024-04-26T21:30:39.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:39.447 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:39.447 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.448+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 msgr2=0x7fdab00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.448+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 0x7fdab00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009b40 tx=0x7fdabc005e50 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.448+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 msgr2=0x7fdacc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.448+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009230 tx=0x7fdab80042f0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 shutdown_connections 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdacc078ed0 0x7fdacc1a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc078530 0x7fdacc19f640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 --2- 172.21.15.97:0/3595810141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdacc0772f0 0x7fdacc19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 >> 172.21.15.97:0/3595810141 conn(0x7fdacc100120 msgr2=0x7fdacc101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 shutdown_connections 2024-04-26T21:30:39.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:39.449+0000 7fdad3200640 1 -- 172.21.15.97:0/3595810141 wait complete. 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:39 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3595810141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:40.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:39 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3595810141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:39 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3595810141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:40.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:40 smithi097 ceph-mon[28127]: Deploying daemon mds.a on smithi097 2024-04-26T21:30:40.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:40 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6838/3674554302,v1:172.21.15.97:6839/3674554302] up:boot 2024-04-26T21:30:40.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:40 smithi097 ceph-mon[28127]: fsmap 10 up:standby 2024-04-26T21:30:40.991 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:30:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:40 smithi111 ceph-mon[31001]: Deploying daemon mds.a on smithi097 2024-04-26T21:30:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:40 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6838/3674554302,v1:172.21.15.97:6839/3674554302] up:boot 2024-04-26T21:30:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:40 smithi111 ceph-mon[31001]: fsmap 10 up:standby 2024-04-26T21:30:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:30:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:40 smithi143 ceph-mon[31461]: Deploying daemon mds.a on smithi097 2024-04-26T21:30:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:40 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6838/3674554302,v1:172.21.15.97:6839/3674554302] up:boot 2024-04-26T21:30:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:40 smithi143 ceph-mon[31461]: fsmap 10 up:standby 2024-04-26T21:30:41.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:30:41.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:41.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 -- 172.21.15.97:0/3132376672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8102950 msgr2=0x7f46f8102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 --2- 172.21.15.97:0/3132376672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8102950 0x7f46f8102d70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009900 tx=0x7f46ec02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 -- 172.21.15.97:0/3132376672 shutdown_connections 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 --2- 172.21.15.97:0/3132376672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 --2- 172.21.15.97:0/3132376672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8103b40 0x7f46f8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 --2- 172.21.15.97:0/3132376672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8102950 0x7f46f8102d70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 -- 172.21.15.97:0/3132376672 >> 172.21.15.97:0/3132376672 conn(0x7f46f80fdee0 msgr2=0x7f46f8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 -- 172.21.15.97:0/3132376672 shutdown_connections 2024-04-26T21:30:41.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.695+0000 7f46ff600640 1 -- 172.21.15.97:0/3132376672 wait complete. 2024-04-26T21:30:41.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.696+0000 7f46ff600640 1 Processor -- start 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 -- start start 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 0x7f46f819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f81a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46f810e300 con 0x7f46f8103b40 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f46f810e180 con 0x7f46f8102950 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46ff600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46f810e480 con 0x7f46f8104500 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56846/0 (socket says 172.21.15.97:56846) 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46f7e00640 1 -- 172.21.15.97:0/4137584013 learned_addr learned my addr 172.21.15.97:0/4137584013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.697+0000 7f46fcc00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 0x7f46f819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 -- 172.21.15.97:0/4137584013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 msgr2=0x7f46f81a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46fd600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f81a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f81a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 -- 172.21.15.97:0/4137584013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 msgr2=0x7f46f819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 0x7f46f819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 -- 172.21.15.97:0/4137584013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f81a2880 con 0x7f46f8103b40 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46fd600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f81a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46fcc00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 0x7f46f819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f7e00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c920 tx=0x7f46e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007c70 con 0x7f46f8103b40 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e4007e10 con 0x7f46f8103b40 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f81a2b70 con 0x7f46f8103b40 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.698+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4018710 con 0x7f46f8103b40 2024-04-26T21:30:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.699+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f81a3110 con 0x7f46f8103b40 2024-04-26T21:30:41.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.700+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f81a3340 con 0x7f46f8103b40 2024-04-26T21:30:41.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.701+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46e400f450 con 0x7f46f8103b40 2024-04-26T21:30:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.704+0000 7f46f5600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 0x7f46e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.704+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f46e40d0260 con 0x7f46f8103b40 2024-04-26T21:30:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.704+0000 7f46fcc00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 0x7f46e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.705+0000 7f46fcc00640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 0x7f46e00a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0057c0 tx=0x7f46ec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.709+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f46e4014030 con 0x7f46f8103b40 2024-04-26T21:30:41.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.975+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46f81a3510 con 0x7f46f8103b40 2024-04-26T21:30:41.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.977+0000 7f46f5600640 1 -- 172.21.15.97:0/4137584013 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f46e409d040 con 0x7f46f8103b40 2024-04-26T21:30:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:41.977 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:41.977 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2024-04-26T21:30:41.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.979+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 msgr2=0x7f46e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:41.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.979+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 0x7f46e00a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0057c0 tx=0x7f46ec031400 comp rx=0 tx=0).stop 2024-04-26T21:30:41.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.979+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 msgr2=0x7f46f819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:41.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.979+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c920 tx=0x7f46e400cde0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 shutdown_connections 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46f8104500 0x7f46f81a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46e00a3c30 0x7f46e00a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f8103b40 0x7f46f819b1b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 --2- 172.21.15.97:0/4137584013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46f8102950 0x7f46f819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 >> 172.21.15.97:0/4137584013 conn(0x7f46f80fdee0 msgr2=0x7f46f81092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 shutdown_connections 2024-04-26T21:30:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:41.980+0000 7f46ff600640 1 -- 172.21.15.97:0/4137584013 wait complete. 2024-04-26T21:30:42.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:41 smithi111 ceph-mon[31001]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:42.232 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:41 smithi097 ceph-mon[28127]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:41 smithi143 ceph-mon[31461]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:42.565 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 systemd[1]: Starting Ceph mds.a for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:42.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:42 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4137584013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:42.895 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 podman[69807]: 2024-04-26 21:30:42.497566263 +0000 UTC m=+0.037370412 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:42.895 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 podman[69807]: 2024-04-26 21:30:42.603929137 +0000 UTC m=+0.143733310 container create 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:30:42.895 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 podman[69807]: 2024-04-26 21:30:42.893734592 +0000 UTC m=+0.433538775 container init 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T21:30:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:42 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4137584013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:43.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:42 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4137584013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 podman[69807]: 2024-04-26 21:30:42.903843186 +0000 UTC m=+0.443647366 container start 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 ceph-mds[69826]: -- 172.21.15.97:0/2572884091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a4af1aa4e0 con 0x55a4af181000 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 ceph-mds[69826]: -- 172.21.15.97:0/2572884091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4af1ada40 con 0x55a4af181000 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: starting mds.a at 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:42 smithi097 bash[69807]: 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 2024-04-26T21:30:43.315 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:30:43 smithi097 systemd[1]: Started Ceph mds.a for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:44.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:44.096 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:44.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:44.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 -- 172.21.15.97:0/3764911691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 msgr2=0x7f973c1113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:44.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 --2- 172.21.15.97:0/3764911691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1113d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9734009ca0 tx=0x7f973402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 -- 172.21.15.97:0/3764911691 shutdown_connections 2024-04-26T21:30:44.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 --2- 172.21.15.97:0/3764911691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1113d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 --2- 172.21.15.97:0/3764911691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c10a180 0x7f973c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 --2- 172.21.15.97:0/3764911691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c108f90 0x7f973c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.226+0000 7f9740e00640 1 -- 172.21.15.97:0/3764911691 >> 172.21.15.97:0/3764911691 conn(0x7f973c077f40 msgr2=0x7f973c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:44.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.227+0000 7f9740e00640 1 -- 172.21.15.97:0/3764911691 shutdown_connections 2024-04-26T21:30:44.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.227+0000 7f9740e00640 1 -- 172.21.15.97:0/3764911691 wait complete. 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.227+0000 7f9740e00640 1 Processor -- start 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 -- start start 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 0x7f973c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c10a180 0x7f973c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f973c1149d0 con 0x7f973c10ab40 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f973c114850 con 0x7f973c10a180 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9740e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f973c114b50 con 0x7f973c108f90 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f973a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f973a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56888/0 (socket says 172.21.15.97:56888) 2024-04-26T21:30:44.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f973a800640 1 -- 172.21.15.97:0/3982312733 learned_addr learned my addr 172.21.15.97:0/3982312733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.228+0000 7f9731400640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c10a180 0x7f973c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9739e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 0x7f973c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 -- 172.21.15.97:0/3982312733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 msgr2=0x7f973c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 0x7f973c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 -- 172.21.15.97:0/3982312733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c10a180 msgr2=0x7f973c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c10a180 0x7f973c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 -- 172.21.15.97:0/3982312733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c1a6ca0 con 0x7f973c10ab40 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9739e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 0x7f973c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f973a800640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9734031040 tx=0x7f9734003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734004240 con 0x7f973c10ab40 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9734003e40 con 0x7f973c10ab40 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c1a6f30 con 0x7f973c10ab40 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.229+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973403b410 con 0x7f973c10ab40 2024-04-26T21:30:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.230+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c1a73f0 con 0x7f973c10ab40 2024-04-26T21:30:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.231+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c110300 con 0x7f973c10ab40 2024-04-26T21:30:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.233+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f97340306e0 con 0x7f973c10ab40 2024-04-26T21:30:44.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.234+0000 7f9733400640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 0x7f97200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:44.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.234+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f97340f1850 con 0x7f973c10ab40 2024-04-26T21:30:44.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.235+0000 7f9739e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 0x7f97200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.235+0000 7f9739e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 0x7f97200a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f973c109fe0 tx=0x7f972c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:44.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.238+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f97340be630 con 0x7f973c10ab40 2024-04-26T21:30:44.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:44.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:44.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:30:44.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:30:44.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.482+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f973c0635d0 con 0x7f973c10ab40 2024-04-26T21:30:44.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.483+0000 7f9733400640 1 -- 172.21.15.97:0/3982312733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f97340bdf30 con 0x7f973c10ab40 2024-04-26T21:30:44.484 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-04-26T21:30:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.486+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 msgr2=0x7f97200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.486+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 0x7f97200a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f973c109fe0 tx=0x7f972c005d30 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.486+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 msgr2=0x7f973c1a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.486+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9734031040 tx=0x7f9734003ce0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 shutdown_connections 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f97200a3bb0 0x7f97200a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c10ab40 0x7f973c1a6630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f973c10a180 0x7f973c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 --2- 172.21.15.97:0/3982312733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c108f90 0x7f973c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 >> 172.21.15.97:0/3982312733 conn(0x7f973c077f40 msgr2=0x7f973c10d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 shutdown_connections 2024-04-26T21:30:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:44.487+0000 7f9740e00640 1 -- 172.21.15.97:0/3982312733 wait complete. 2024-04-26T21:30:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: Deploying daemon mds.b on smithi111 2024-04-26T21:30:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6840/1900627986,v1:172.21.15.97:6841/1900627986] up:boot 2024-04-26T21:30:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: fsmap 11 up:standby 2024-04-26T21:30:45.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:30:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3982312733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:45.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:44 smithi111 ceph-mon[31001]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: Deploying daemon mds.b on smithi111 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6840/1900627986,v1:172.21.15.97:6841/1900627986] up:boot 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: fsmap 11 up:standby 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3982312733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:45.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:44 smithi143 ceph-mon[31461]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: Deploying daemon mds.b on smithi111 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6840/1900627986,v1:172.21.15.97:6841/1900627986] up:boot 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: fsmap 11 up:standby 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3982312733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:44 smithi097 ceph-mon[28127]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:46.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.724+0000 7f4653a00640 1 -- 172.21.15.97:0/907299739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c101b80 msgr2=0x7f464c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.724+0000 7f4653a00640 1 --2- 172.21.15.97:0/907299739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c101b80 0x7f464c101fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4638009900 tx=0x7f463802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 -- 172.21.15.97:0/907299739 shutdown_connections 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 --2- 172.21.15.97:0/907299739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c102520 0x7f464c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 --2- 172.21.15.97:0/907299739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c101b80 0x7f464c101fe0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 --2- 172.21.15.97:0/907299739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 -- 172.21.15.97:0/907299739 >> 172.21.15.97:0/907299739 conn(0x7f464c0fce00 msgr2=0x7f464c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 -- 172.21.15.97:0/907299739 shutdown_connections 2024-04-26T21:30:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.725+0000 7f4653a00640 1 -- 172.21.15.97:0/907299739 wait complete. 2024-04-26T21:30:46.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 Processor -- start 2024-04-26T21:30:46.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 -- start start 2024-04-26T21:30:46.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c101b80 0x7f464c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1104f0 con 0x7f464c102520 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f464c110370 con 0x7f464c101b80 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.726+0000 7f4653a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f464c110670 con 0x7f464c101220 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56904/0 (socket says 172.21.15.97:56904) 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 -- 172.21.15.97:0/173801481 learned_addr learned my addr 172.21.15.97:0/173801481 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f464be00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c101b80 0x7f464c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:46.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651000640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 -- 172.21.15.97:0/173801481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 msgr2=0x7f464c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 -- 172.21.15.97:0/173801481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c101b80 msgr2=0x7f464c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c101b80 0x7f464c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 -- 172.21.15.97:0/173801481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c1a28d0 con 0x7f464c102520 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651000640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:46.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.727+0000 7f4651a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f464400eb10 tx=0x7f464400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.728+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644004b20 con 0x7f464c102520 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.728+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4644004cc0 con 0x7f464c102520 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.728+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c1a2bc0 con 0x7f464c102520 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.728+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644010690 con 0x7f464c102520 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.728+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1a3090 con 0x7f464c102520 2024-04-26T21:30:46.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.729+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c10c240 con 0x7f464c102520 2024-04-26T21:30:46.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.731+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4644005020 con 0x7f464c102520 2024-04-26T21:30:46.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.732+0000 7f4649600640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 0x7f46340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:46.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.733+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f46440d06f0 con 0x7f464c102520 2024-04-26T21:30:46.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.733+0000 7f4651000640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 0x7f46340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:46.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.733+0000 7f4651000640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 0x7f46340a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f464c19bb60 tx=0x7f4640005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:46.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.737+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f464409d4d0 con 0x7f464c102520 2024-04-26T21:30:46.796 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:46 smithi111 systemd[1]: Starting Ceph mds.b for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:30:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.976+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f464c02dad0 con 0x7f464c102520 2024-04-26T21:30:46.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.977+0000 7f4649600640 1 -- 172.21.15.97:0/173801481 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f464409cdd0 con 0x7f464c102520 2024-04-26T21:30:46.979 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:46.979 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:30:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 msgr2=0x7f46340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 0x7f46340a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f464c19bb60 tx=0x7f4640005e70 comp rx=0 tx=0).stop 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 msgr2=0x7f464c1a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f464400eb10 tx=0x7f464400c2d0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 shutdown_connections 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f46340a3bb0 0x7f46340a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f464c102520 0x7f464c1a2230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f464c101b80 0x7f464c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 --2- 172.21.15.97:0/173801481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f464c101220 0x7f464c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 >> 172.21.15.97:0/173801481 conn(0x7f464c0fce00 msgr2=0x7f464c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 shutdown_connections 2024-04-26T21:30:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:46.980+0000 7f4653a00640 1 -- 172.21.15.97:0/173801481 wait complete. 2024-04-26T21:30:47.067 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:46 smithi111 podman[61237]: 2024-04-26 21:30:46.699902586 +0000 UTC m=+0.032657446 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:30:47.068 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:46 smithi111 podman[61237]: 2024-04-26 21:30:46.821648256 +0000 UTC m=+0.154403155 container create 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T21:30:47.068 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 podman[61237]: 2024-04-26 21:30:47.066697649 +0000 UTC m=+0.399452588 container init 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T21:30:47.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 podman[61237]: 2024-04-26 21:30:47.078358901 +0000 UTC m=+0.411113767 container start 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:30:47.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mds[61256]: -- 172.21.15.111:0/3684436519 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570a590ba40 con 0x5570a58df000 2024-04-26T21:30:47.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: starting mds.b at 2024-04-26T21:30:47.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 bash[61237]: 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b 2024-04-26T21:30:47.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:47 smithi111 systemd[1]: Started Ceph mds.b for 9ce681f8-0412-11ef-bc93-c7b262605968. 2024-04-26T21:30:48.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/173801481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:47 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:48.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/173801481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:47 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/173801481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:48.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:47 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:49.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:30:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:48 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:boot 2024-04-26T21:30:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:48 smithi143 ceph-mon[31461]: fsmap 12 up:standby 2024-04-26T21:30:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:30:49.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:48 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:boot 2024-04-26T21:30:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:48 smithi097 ceph-mon[28127]: fsmap 12 up:standby 2024-04-26T21:30:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:30:49.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:48 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:boot 2024-04-26T21:30:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:48 smithi111 ceph-mon[31001]: fsmap 12 up:standby 2024-04-26T21:30:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:30:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.217+0000 7f6c1de00640 1 -- 172.21.15.97:0/3445670688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 msgr2=0x7f6c18108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.217+0000 7f6c1de00640 1 --2- 172.21.15.97:0/3445670688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18108100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009900 tx=0x7f6c0402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 -- 172.21.15.97:0/3445670688 shutdown_connections 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 --2- 172.21.15.97:0/3445670688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c18108640 0x7f6c1810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 --2- 172.21.15.97:0/3445670688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18108100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 --2- 172.21.15.97:0/3445670688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c181033e0 0x7f6c181057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 -- 172.21.15.97:0/3445670688 >> 172.21.15.97:0/3445670688 conn(0x7f6c180fcdf0 msgr2=0x7f6c180ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 -- 172.21.15.97:0/3445670688 shutdown_connections 2024-04-26T21:30:49.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.218+0000 7f6c1de00640 1 -- 172.21.15.97:0/3445670688 wait complete. 2024-04-26T21:30:49.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.219+0000 7f6c1de00640 1 Processor -- start 2024-04-26T21:30:49.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.219+0000 7f6c1de00640 1 -- start start 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c181033e0 0x7f6c18198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c18108640 0x7f6c181a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810e380 con 0x7f6c18105d10 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810e200 con 0x7f6c181033e0 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c1de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810e500 con 0x7f6c18108640 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c16000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c16000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56920/0 (socket says 172.21.15.97:56920) 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c16000640 1 -- 172.21.15.97:0/2779996304 learned_addr learned my addr 172.21.15.97:0/2779996304 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c16a00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c181033e0 0x7f6c18198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.220+0000 7f6c17400640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c18108640 0x7f6c181a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 -- 172.21.15.97:0/2779996304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c18108640 msgr2=0x7f6c181a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c18108640 0x7f6c181a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 -- 172.21.15.97:0/2779996304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c181033e0 msgr2=0x7f6c18198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c181033e0 0x7f6c18198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 -- 172.21.15.97:0/2779996304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c181a06b0 con 0x7f6c18105d10 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.221+0000 7f6c16000640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c04031460 tx=0x7f6c04003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.222+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0403f070 con 0x7f6c18105d10 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.222+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c04031da0 con 0x7f6c18105d10 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.222+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04035030 con 0x7f6c18105d10 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.222+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c181a0940 con 0x7f6c18105d10 2024-04-26T21:30:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.222+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c181a0de0 con 0x7f6c18105d10 2024-04-26T21:30:49.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.223+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c180fe8e0 con 0x7f6c18105d10 2024-04-26T21:30:49.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.225+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c04042070 con 0x7f6c18105d10 2024-04-26T21:30:49.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.228+0000 7f6c1ca00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 0x7f6bfc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.228+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6c040f0e20 con 0x7f6c18105d10 2024-04-26T21:30:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.228+0000 7f6c16a00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 0x7f6bfc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.229+0000 7f6c16a00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 0x7f6bfc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005d00 tx=0x7f6c0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.230+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6c040bd5b0 con 0x7f6c18105d10 2024-04-26T21:30:49.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.474+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c1802dc90 con 0x7f6c18105d10 2024-04-26T21:30:49.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.475+0000 7f6c1ca00640 1 -- 172.21.15.97:0/2779996304 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f6c040c18b0 con 0x7f6c18105d10 2024-04-26T21:30:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:49.476 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34433,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:30:49.476 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 13 2024-04-26T21:30:49.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.477+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 msgr2=0x7f6bfc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.477+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 0x7f6bfc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005d00 tx=0x7f6c0800b410 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.477+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 msgr2=0x7f6c18198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.477+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c04031460 tx=0x7f6c04003c80 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 shutdown_connections 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c18108640 0x7f6c181a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6bfc0a3c30 0x7f6bfc0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18105d10 0x7f6c18198fd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 --2- 172.21.15.97:0/2779996304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c181033e0 0x7f6c18198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 >> 172.21.15.97:0/2779996304 conn(0x7f6c180fcdf0 msgr2=0x7f6c18103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 shutdown_connections 2024-04-26T21:30:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.478+0000 7f6c1de00640 1 -- 172.21.15.97:0/2779996304 wait complete. 2024-04-26T21:30:49.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:30:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 -- 172.21.15.97:0/2768923757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b88108650 msgr2=0x7f7b8810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 --2- 172.21.15.97:0/2768923757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b88108650 0x7f7b8810aaa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009ca0 tx=0x7f7b8002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 -- 172.21.15.97:0/2768923757 shutdown_connections 2024-04-26T21:30:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 --2- 172.21.15.97:0/2768923757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b88108650 0x7f7b8810aaa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 --2- 172.21.15.97:0/2768923757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 0x7f7b88108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 --2- 172.21.15.97:0/2768923757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b881033f0 0x7f7b881057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 -- 172.21.15.97:0/2768923757 >> 172.21.15.97:0/2768923757 conn(0x7f7b880fcde0 msgr2=0x7f7b880ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 -- 172.21.15.97:0/2768923757 shutdown_connections 2024-04-26T21:30:49.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.684+0000 7f7b8f800640 1 -- 172.21.15.97:0/2768923757 wait complete. 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 Processor -- start 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 -- start start 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 0x7f7b8819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b88108650 0x7f7b881a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e3c0 con 0x7f7b881033f0 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e240 con 0x7f7b88105d20 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8f800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e540 con 0x7f7b88108650 2024-04-26T21:30:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.685+0000 7f7b8ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56940/0 (socket says 172.21.15.97:56940) 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 -- 172.21.15.97:0/1114641955 learned_addr learned my addr 172.21.15.97:0/1114641955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b87e00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 0x7f7b8819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 -- 172.21.15.97:0/1114641955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b88108650 msgr2=0x7f7b881a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b88108650 0x7f7b881a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 -- 172.21.15.97:0/1114641955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 msgr2=0x7f7b8819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 0x7f7b8819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 -- 172.21.15.97:0/1114641955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881a2900 con 0x7f7b881033f0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8ce00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d770 tx=0x7f7b7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c017070 con 0x7f7b881033f0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b7c0054b0 con 0x7f7b881033f0 2024-04-26T21:30:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c004c60 con 0x7f7b881033f0 2024-04-26T21:30:49.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.686+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a2bf0 con 0x7f7b881033f0 2024-04-26T21:30:49.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.687+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a30f0 con 0x7f7b881033f0 2024-04-26T21:30:49.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.688+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8810a140 con 0x7f7b881033f0 2024-04-26T21:30:49.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.690+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b7c004e00 con 0x7f7b881033f0 2024-04-26T21:30:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.692+0000 7f7b85600640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 0x7f7b700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.692+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7b7c0cfa50 con 0x7f7b881033f0 2024-04-26T21:30:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.692+0000 7f7b87e00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 0x7f7b700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.693+0000 7f7b87e00640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 0x7f7b700a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005d00 tx=0x7f7b7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.697+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7b7c09c830 con 0x7f7b881033f0 2024-04-26T21:30:49.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:49 smithi097 ceph-mon[28127]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:49.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:49 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2779996304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.909+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b8802dd10 con 0x7f7b881033f0 2024-04-26T21:30:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.910+0000 7f7b85600640 1 -- 172.21.15.97:0/1114641955 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18457 (secure 0 0 0) 0x7f7b7c09c130 con 0x7f7b881033f0 2024-04-26T21:30:49.911 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:49.911 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:29:48.485798+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.912+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 msgr2=0x7f7b700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.912+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 0x7f7b700a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005d00 tx=0x7f7b7400b410 comp rx=0 tx=0).stop 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.912+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 msgr2=0x7f7b8819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.912+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d770 tx=0x7f7b7c00dc30 comp rx=0 tx=0).stop 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 shutdown_connections 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b88108650 0x7f7b881a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b88105d20 0x7f7b8819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7b700a3c30 0x7f7b700a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 --2- 172.21.15.97:0/1114641955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b881033f0 0x7f7b8819acf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:49.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 >> 172.21.15.97:0/1114641955 conn(0x7f7b880fcde0 msgr2=0x7f7b88102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:49.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 shutdown_connections 2024-04-26T21:30:49.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:49.913+0000 7f7b8f800640 1 -- 172.21.15.97:0/1114641955 wait complete. 2024-04-26T21:30:49.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:30:49.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:30:50.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:49 smithi143 ceph-mon[31461]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:50.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:49 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2779996304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:50.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 -- 172.21.15.97:0/3649635298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 msgr2=0x7fa3a0103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 --2- 172.21.15.97:0/3649635298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a0103ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009900 tx=0x7fa38c02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 -- 172.21.15.97:0/3649635298 shutdown_connections 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 --2- 172.21.15.97:0/3649635298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 0x7fa3a010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 --2- 172.21.15.97:0/3649635298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a0103ff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 --2- 172.21.15.97:0/3649635298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 0x7fa3a0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 -- 172.21.15.97:0/3649635298 >> 172.21.15.97:0/3649635298 conn(0x7fa3a00fdf10 msgr2=0x7fa3a0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 -- 172.21.15.97:0/3649635298 shutdown_connections 2024-04-26T21:30:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.120+0000 7fa3a6000640 1 -- 172.21.15.97:0/3649635298 wait complete. 2024-04-26T21:30:50.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 Processor -- start 2024-04-26T21:30:50.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 -- start start 2024-04-26T21:30:50.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 0x7fa3a019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 0x7fa3a01a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010e360 con 0x7fa3a0103b70 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010e1e0 con 0x7fa3a0104530 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa3a6000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a010e4e0 con 0x7fa3a0102980 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56962/0 (socket says 172.21.15.97:56962) 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa39ea00640 1 -- 172.21.15.97:0/2613991880 learned_addr learned my addr 172.21.15.97:0/2613991880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.122+0000 7fa39f400640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 0x7fa3a019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39fe00640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 0x7fa3a01a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 -- 172.21.15.97:0/2613991880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 msgr2=0x7fa3a019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 0x7fa3a019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 -- 172.21.15.97:0/2613991880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 msgr2=0x7fa3a01a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 0x7fa3a01a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 -- 172.21.15.97:0/2613991880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a01a2910 con 0x7fa3a0103b70 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa39ea00640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009a20 tx=0x7fa38c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c03f030 con 0x7fa3a0103b70 2024-04-26T21:30:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa38c003a80 con 0x7fa3a0103b70 2024-04-26T21:30:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c035070 con 0x7fa3a0103b70 2024-04-26T21:30:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.123+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01a2ba0 con 0x7fa3a0103b70 2024-04-26T21:30:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.124+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a3090 con 0x7fa3a0103b70 2024-04-26T21:30:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.125+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a0195150 con 0x7fa3a0103b70 2024-04-26T21:30:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.127+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa38c042030 con 0x7fa3a0103b70 2024-04-26T21:30:50.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.128+0000 7fa397e00640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 0x7fa3840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.129+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa38c0f05d0 con 0x7fa3a0103b70 2024-04-26T21:30:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.129+0000 7fa39f400640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 0x7fa3840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.130+0000 7fa39f400640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 0x7fa3840a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3a019c1d0 tx=0x7fa390005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:50.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.133+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa38c0bcd60 con 0x7fa3a0103b70 2024-04-26T21:30:50.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:49 smithi111 ceph-mon[31001]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:49 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2779996304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:30:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.341+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3a002dad0 con 0x7fa3a0103b70 2024-04-26T21:30:50.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.341+0000 7fa397e00640 1 -- 172.21.15.97:0/2613991880 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fa38c0c1060 con 0x7fa3a0103b70 2024-04-26T21:30:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:30:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.343+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 msgr2=0x7fa3840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.343+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 0x7fa3840a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3a019c1d0 tx=0x7fa390005f90 comp rx=0 tx=0).stop 2024-04-26T21:30:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.343+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 msgr2=0x7fa3a019b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.343+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009a20 tx=0x7fa38c0040c0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 shutdown_connections 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3a0104530 0x7fa3a01a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa3840a3bb0 0x7fa3840a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3a0103b70 0x7fa3a019b240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 --2- 172.21.15.97:0/2613991880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3a0102980 0x7fa3a019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 >> 172.21.15.97:0/2613991880 conn(0x7fa3a00fdf10 msgr2=0x7fa3a0109430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 shutdown_connections 2024-04-26T21:30:50.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.344+0000 7fa3a6000640 1 -- 172.21.15.97:0/2613991880 wait complete. 2024-04-26T21:30:50.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:30:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 -- 172.21.15.97:0/742773684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104102980 msgr2=0x7f1104102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 --2- 172.21.15.97:0/742773684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104102980 0x7f1104102da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009900 tx=0x7f10f402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 -- 172.21.15.97:0/742773684 shutdown_connections 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 --2- 172.21.15.97:0/742773684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104104530 0x7f110410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 --2- 172.21.15.97:0/742773684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 0x7f1104103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 --2- 172.21.15.97:0/742773684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104102980 0x7f1104102da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 -- 172.21.15.97:0/742773684 >> 172.21.15.97:0/742773684 conn(0x7f11040fdf10 msgr2=0x7f1104100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.539+0000 7f110b000640 1 -- 172.21.15.97:0/742773684 shutdown_connections 2024-04-26T21:30:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.540+0000 7f110b000640 1 -- 172.21.15.97:0/742773684 wait complete. 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.540+0000 7f110b000640 1 Processor -- start 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 -- start start 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104102980 0x7f110419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 0x7f110419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f110410e3a0 con 0x7f1104104530 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f110410e220 con 0x7f1104102980 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f110b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f110410e520 con 0x7f1104103b70 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f1103e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104102980 0x7f110419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f1109000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f1109000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56982/0 (socket says 172.21.15.97:56982) 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f1109000640 1 -- 172.21.15.97:0/559974784 learned_addr learned my addr 172.21.15.97:0/559974784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.541+0000 7f1103400640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 0x7f110419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 -- 172.21.15.97:0/559974784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 msgr2=0x7f110419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 0x7f110419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 -- 172.21.15.97:0/559974784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104102980 msgr2=0x7f110419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104102980 0x7f110419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 -- 172.21.15.97:0/559974784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11041a2930 con 0x7f1104104530 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1109000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00eaa0 tx=0x7f10fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00cdb0 con 0x7f1104104530 2024-04-26T21:30:50.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10fc005020 con 0x7f1104104530 2024-04-26T21:30:50.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11041a2c20 con 0x7f1104104530 2024-04-26T21:30:50.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.542+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0116d0 con 0x7f1104104530 2024-04-26T21:30:50.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.543+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041a3140 con 0x7f1104104530 2024-04-26T21:30:50.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.544+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11041a3310 con 0x7f1104104530 2024-04-26T21:30:50.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.545+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10fc0051d0 con 0x7f1104104530 2024-04-26T21:30:50.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.547+0000 7f1100c00640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 0x7f10e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.547+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f10fc0d0170 con 0x7f1104104530 2024-04-26T21:30:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.547+0000 7f1103e00640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 0x7f10e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.548+0000 7f1103e00640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 0x7f10e80a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10f40057c0 tx=0x7f10f4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:50.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.551+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f10fc016030 con 0x7f1104104530 2024-04-26T21:30:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.771+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11041a3580 con 0x7f1104104530 2024-04-26T21:30:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.772+0000 7f1100c00640 1 -- 172.21.15.97:0/559974784 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f10fc09c900 con 0x7f1104104530 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-26T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.775+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 msgr2=0x7f10e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.775+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 0x7f10e80a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10f40057c0 tx=0x7f10f4005a20 comp rx=0 tx=0).stop 2024-04-26T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.775+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 msgr2=0x7f11041a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.775+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00eaa0 tx=0x7f10fc00ef60 comp rx=0 tx=0).stop 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 shutdown_connections 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10e80a3c30 0x7f10e80a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1104104530 0x7f11041a2290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1104103b70 0x7f110419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 --2- 172.21.15.97:0/559974784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1104102980 0x7f110419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 >> 172.21.15.97:0/559974784 conn(0x7f11040fdf10 msgr2=0x7f1104109340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 shutdown_connections 2024-04-26T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.776+0000 7f110b000640 1 -- 172.21.15.97:0/559974784 wait complete. 2024-04-26T21:30:50.801 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:30:50.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:30:50.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 -- 172.21.15.97:0/1942562667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d14103470 msgr2=0x7f8d14105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 --2- 172.21.15.97:0/1942562667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d14103470 0x7f8d14105860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d0402a410 comp rx=0 tx=0).stop 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 -- 172.21.15.97:0/1942562667 shutdown_connections 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 --2- 172.21.15.97:0/1942562667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d141086d0 0x7f8d1410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 --2- 172.21.15.97:0/1942562667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14105da0 0x7f8d14108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 --2- 172.21.15.97:0/1942562667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d14103470 0x7f8d14105860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 -- 172.21.15.97:0/1942562667 >> 172.21.15.97:0/1942562667 conn(0x7f8d140fcde0 msgr2=0x7f8d140ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 -- 172.21.15.97:0/1942562667 shutdown_connections 2024-04-26T21:30:50.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.993+0000 7f8d19800640 1 -- 172.21.15.97:0/1942562667 wait complete. 2024-04-26T21:30:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.994+0000 7f8d19800640 1 Processor -- start 2024-04-26T21:30:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 -- start start 2024-04-26T21:30:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14103470 0x7f8d14198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d14105da0 0x7f8d14199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410e440 con 0x7f8d141086d0 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410e2c0 con 0x7f8d14105da0 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d19800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1410e5c0 con 0x7f8d14103470 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d13400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d13400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57822/0 (socket says 172.21.15.97:57822) 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d13400640 1 -- 172.21.15.97:0/2896839709 learned_addr learned my addr 172.21.15.97:0/2896839709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d12a00640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14103470 0x7f8d14198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.995+0000 7f8d0a000640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d14105da0 0x7f8d14199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 -- 172.21.15.97:0/2896839709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14103470 msgr2=0x7f8d14198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14103470 0x7f8d14198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 -- 172.21.15.97:0/2896839709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d14105da0 msgr2=0x7f8d14199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d14105da0 0x7f8d14199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 -- 172.21.15.97:0/2896839709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d141a07e0 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d13400640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00b970 tx=0x7f8d0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c019070 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d0c014440 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d141a0ad0 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0075d0 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.996+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a1050 con 0x7f8d141086d0 2024-04-26T21:30:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:50.997+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d140fe8d0 con 0x7f8d141086d0 2024-04-26T21:30:51.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.001+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8d0c01c070 con 0x7f8d141086d0 2024-04-26T21:30:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.002+0000 7f8d0be00640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 0x7f8cf80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.002+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8d0c0cfa80 con 0x7f8d141086d0 2024-04-26T21:30:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.002+0000 7f8d12a00640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 0x7f8cf80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.003+0000 7f8d12a00640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 0x7f8cf80a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006010 tx=0x7f8d04031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:51.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.005+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8d0c09c900 con 0x7f8d141086d0 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1114641955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2613991880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1114641955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2613991880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1114641955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2613991880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:51.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.212+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8d1402dad0 con 0x7f8d141086d0 2024-04-26T21:30:51.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.758+0000 7f8d0be00640 1 -- 172.21.15.97:0/2896839709 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f8d0c09c200 con 0x7f8d141086d0 2024-04-26T21:30:51.759 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-26T21:30:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.763+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 msgr2=0x7f8cf80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.763+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 0x7f8cf80a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006010 tx=0x7f8d04031400 comp rx=0 tx=0).stop 2024-04-26T21:30:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.763+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 msgr2=0x7f8d141a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.763+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00b970 tx=0x7f8d0c00be30 comp rx=0 tx=0).stop 2024-04-26T21:30:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 shutdown_connections 2024-04-26T21:30:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8cf80a3bb0 0x7f8cf80a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d141086d0 0x7f8d141a0170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d14105da0 0x7f8d14199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 --2- 172.21.15.97:0/2896839709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d14103470 0x7f8d14198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 >> 172.21.15.97:0/2896839709 conn(0x7f8d140fcde0 msgr2=0x7f8d14105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.764+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 shutdown_connections 2024-04-26T21:30:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.765+0000 7f8d19800640 1 -- 172.21.15.97:0/2896839709 wait complete. 2024-04-26T21:30:51.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 -- 172.21.15.97:0/569909171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810104520 msgr2=0x7f681010adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 --2- 172.21.15.97:0/569909171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810104520 0x7f681010adb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6808009900 tx=0x7f680802a410 comp rx=0 tx=0).stop 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 -- 172.21.15.97:0/569909171 shutdown_connections 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 --2- 172.21.15.97:0/569909171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810104520 0x7f681010adb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 --2- 172.21.15.97:0/569909171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810103b60 0x7f6810103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.989+0000 7f6818000640 1 --2- 172.21.15.97:0/569909171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f6810102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.990+0000 7f6818000640 1 -- 172.21.15.97:0/569909171 >> 172.21.15.97:0/569909171 conn(0x7f68100fdee0 msgr2=0x7f6810100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.990+0000 7f6818000640 1 -- 172.21.15.97:0/569909171 shutdown_connections 2024-04-26T21:30:51.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.990+0000 7f6818000640 1 -- 172.21.15.97:0/569909171 wait complete. 2024-04-26T21:30:51.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 Processor -- start 2024-04-26T21:30:51.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 -- start start 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f681019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 0x7f68101a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f681010e440 con 0x7f6810103b60 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f681010e2c0 con 0x7f6810102970 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6818000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f681010e5c0 con 0x7f6810104520 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6814c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.991+0000 7f6814c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57830/0 (socket says 172.21.15.97:57830) 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 -- 172.21.15.97:0/2714738548 learned_addr learned my addr 172.21.15.97:0/2714738548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6816000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 0x7f68101a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6815600640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f681019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 -- 172.21.15.97:0/2714738548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 msgr2=0x7f68101a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 0x7f68101a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 -- 172.21.15.97:0/2714738548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 msgr2=0x7f681019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f681019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 -- 172.21.15.97:0/2714738548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68101a2a90 con 0x7f6810103b60 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6816000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 0x7f68101a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6815600640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f681019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6814c00640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00b4a0 tx=0x7f67fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc014070 con 0x7f6810103b60 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67fc010aa0 con 0x7f6810103b60 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc017b70 con 0x7f6810103b60 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.992+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68101a2d80 con 0x7f6810103b60 2024-04-26T21:30:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.993+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a32f0 con 0x7f6810103b60 2024-04-26T21:30:51.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.994+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68101a3580 con 0x7f6810103b60 2024-04-26T21:30:51.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.997+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67fc017070 con 0x7f6810103b60 2024-04-26T21:30:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.998+0000 7f680e000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 0x7f67f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.998+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f67fc0cf830 con 0x7f6810103b60 2024-04-26T21:30:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.999+0000 7f6815600640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 0x7f67f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:51.999+0000 7f6815600640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 0x7f67f80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f68101039c0 tx=0x7f68040073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:52.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.002+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f67fc09c540 con 0x7f6810103b60 2024-04-26T21:30:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:51 smithi143 ceph-mon[31461]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:51 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/559974784' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:30:52.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:51 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:30:52.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:51 smithi097 ceph-mon[28127]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:52.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:51 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/559974784' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:30:52.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:51 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:30:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:51 smithi111 ceph-mon[31001]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:51 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/559974784' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:30:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:51 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:30:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.209+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f68101a3750 con 0x7f6810103b60 2024-04-26T21:30:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.765+0000 7f680e000640 1 -- 172.21.15.97:0/2714738548 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f67fc09be40 con 0x7f6810103b60 2024-04-26T21:30:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-26T21:30:52.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 msgr2=0x7f67f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:52.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 0x7f67f80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f68101039c0 tx=0x7f68040073c0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 msgr2=0x7f681019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:52.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00b4a0 tx=0x7f67fc00b960 comp rx=0 tx=0).stop 2024-04-26T21:30:52.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 shutdown_connections 2024-04-26T21:30:52.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6810104520 0x7f68101a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67f80a3bb0 0x7f67f80a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6810103b60 0x7f681019b3b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 --2- 172.21.15.97:0/2714738548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6810102970 0x7f681019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 >> 172.21.15.97:0/2714738548 conn(0x7f68100fdee0 msgr2=0x7f6810109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.769+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 shutdown_connections 2024-04-26T21:30:52.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.770+0000 7f6818000640 1 -- 172.21.15.97:0/2714738548 wait complete. 2024-04-26T21:30:52.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:30:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 -- 172.21.15.97:0/3598984548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd008075310 msgr2=0x7fd008075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3598984548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd008075310 0x7fd008075770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009900 tx=0x7fcff402a480 comp rx=0 tx=0).stop 2024-04-26T21:30:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 -- 172.21.15.97:0/3598984548 shutdown_connections 2024-04-26T21:30:52.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3598984548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00810a830 0x7fd00810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3598984548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd008075310 0x7fd008075770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3598984548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 0x7fd008074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.984+0000 7fd00fc00640 1 -- 172.21.15.97:0/3598984548 >> 172.21.15.97:0/3598984548 conn(0x7fd0080fdf10 msgr2=0x7fd008100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.985+0000 7fd00fc00640 1 -- 172.21.15.97:0/3598984548 shutdown_connections 2024-04-26T21:30:52.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.985+0000 7fd00fc00640 1 -- 172.21.15.97:0/3598984548 wait complete. 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.985+0000 7fd00fc00640 1 Processor -- start 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 -- start start 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd008075310 0x7fd00819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 0x7fd00819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:52.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd008110570 con 0x7fd00810a830 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081103f0 con 0x7fd0080768f0 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081106f0 con 0x7fd008075310 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57854/0 (socket says 172.21.15.97:57854) 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00dc00640 1 -- 172.21.15.97:0/3553714969 learned_addr learned my addr 172.21.15.97:0/3553714969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:52.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.986+0000 7fd00d200640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd008075310 0x7fd00819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd007e00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 0x7fd00819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 -- 172.21.15.97:0/3553714969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd008075310 msgr2=0x7fd00819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd008075310 0x7fd00819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 -- 172.21.15.97:0/3553714969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 msgr2=0x7fd00819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 0x7fd00819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 -- 172.21.15.97:0/3553714969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0081a2930 con 0x7fd00810a830 2024-04-26T21:30:52.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00dc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd00000eaa0 tx=0x7fd00000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:52.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cdb0 con 0x7fd00810a830 2024-04-26T21:30:52.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd000005020 con 0x7fd00810a830 2024-04-26T21:30:52.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0081a2c20 con 0x7fd00810a830 2024-04-26T21:30:52.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.987+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000116d0 con 0x7fd00810a830 2024-04-26T21:30:52.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.988+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081a3170 con 0x7fd00810a830 2024-04-26T21:30:52.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.989+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0081095f0 con 0x7fd00810a830 2024-04-26T21:30:52.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.992+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0000051d0 con 0x7fd00810a830 2024-04-26T21:30:52.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.994+0000 7fd005600640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 0x7fcff00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:52.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.994+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7fd0000d0830 con 0x7fd00810a830 2024-04-26T21:30:52.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.994+0000 7fd00d200640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 0x7fcff00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:52.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.995+0000 7fd00d200640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 0x7fcff00a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd00819c1f0 tx=0x7fcffc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:52.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:52.996+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd000016030 con 0x7fd00810a830 2024-04-26T21:30:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:30:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:30:53.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:53.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:53.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:53.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:30:53.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:30:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2896839709' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:30:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:53.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.243+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd00802dad0 con 0x7fd00810a830 2024-04-26T21:30:53.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.953+0000 7fd005600640 1 -- 172.21.15.97:0/3553714969 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd00009d3c0 con 0x7fd00810a830 2024-04-26T21:30:53.954 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.956+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 msgr2=0x7fcff00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.956+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 0x7fcff00a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd00819c1f0 tx=0x7fcffc00b390 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 msgr2=0x7fd0081a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd00000eaa0 tx=0x7fd00000ef60 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 shutdown_connections 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcff00a3bb0 0x7fcff00a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00810a830 0x7fd0081a2290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0080768f0 0x7fd00819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 --2- 172.21.15.97:0/3553714969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd008075310 0x7fd00819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 >> 172.21.15.97:0/3553714969 conn(0x7fd0080fdf10 msgr2=0x7fd0081067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 shutdown_connections 2024-04-26T21:30:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:53.957+0000 7fd00fc00640 1 -- 172.21.15.97:0/3553714969 wait complete. 2024-04-26T21:30:53.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: pgmap v361: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:30:54.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:53 smithi143 ceph-mon[31461]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:30:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: pgmap v361: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:30:54.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:30:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:30:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:30:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:53 smithi097 ceph-mon[28127]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:30:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: pgmap v361: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2714738548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:30:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:30:54.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:30:54.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:30:54.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:53 smithi111 ceph-mon[31001]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 -- 172.21.15.97:0/2470521351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 msgr2=0x7fea0c104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 --2- 172.21.15.97:0/2470521351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c104d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009900 tx=0x7fe9fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 -- 172.21.15.97:0/2470521351 shutdown_connections 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 --2- 172.21.15.97:0/2470521351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c106510 0x7fea0c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 --2- 172.21.15.97:0/2470521351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea0c105b50 0x7fea0c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 --2- 172.21.15.97:0/2470521351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c104d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 -- 172.21.15.97:0/2470521351 >> 172.21.15.97:0/2470521351 conn(0x7fea0c0ffeb0 msgr2=0x7fea0c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 -- 172.21.15.97:0/2470521351 shutdown_connections 2024-04-26T21:30:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.157+0000 7fea11400640 1 -- 172.21.15.97:0/2470521351 wait complete. 2024-04-26T21:30:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.158+0000 7fea11400640 1 Processor -- start 2024-04-26T21:30:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.158+0000 7fea11400640 1 -- start start 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 0x7fea0c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea0c106510 0x7fea0c1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c110360 con 0x7fea0c104960 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1101e0 con 0x7fea0c105b50 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea11400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1104e0 con 0x7fea0c106510 2024-04-26T21:30:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57876/0 (socket says 172.21.15.97:57876) 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 -- 172.21.15.97:0/319683325 learned_addr learned my addr 172.21.15.97:0/319683325 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea03400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 0x7fea0c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 -- 172.21.15.97:0/319683325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea0c106510 msgr2=0x7fea0c1a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea0c106510 0x7fea0c1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 -- 172.21.15.97:0/319683325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 msgr2=0x7fea0c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 0x7fea0c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.159+0000 7fea0be00640 1 -- 172.21.15.97:0/319683325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c1a2650 con 0x7fea0c104960 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea03400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 0x7fea0c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea0be00640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc031460 tx=0x7fe9fc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc03f070 con 0x7fea0c104960 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c1a28e0 con 0x7fea0c104960 2024-04-26T21:30:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9fc031da0 con 0x7fea0c104960 2024-04-26T21:30:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc035030 con 0x7fea0c104960 2024-04-26T21:30:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.160+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1a2dd0 con 0x7fea0c104960 2024-04-26T21:30:54.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.162+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0c10bda0 con 0x7fea0c104960 2024-04-26T21:30:54.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.164+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe9fc042070 con 0x7fea0c104960 2024-04-26T21:30:54.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.165+0000 7fea09600640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.166+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe9fc0f0cb0 con 0x7fea0c104960 2024-04-26T21:30:54.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.166+0000 7fea03400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.166+0000 7fea03400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea0c19bf40 tx=0x7fe9f4005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:54.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.170+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe9fc0bd740 con 0x7fea0c104960 2024-04-26T21:30:54.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.374+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea0c0635d0 con 0x7fea0c104960 2024-04-26T21:30:54.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.377+0000 7fea09600640 1 -- 172.21.15.97:0/319683325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22233 (secure 0 0 0) 0x7fe9fc0bd040 con 0x7fea0c104960 2024-04-26T21:30:54.378 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:54.379 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:30:53.245360+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 msgr2=0x7fe9f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea0c19bf40 tx=0x7fe9f4005d90 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 msgr2=0x7fea0c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc031460 tx=0x7fe9fc003c80 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 shutdown_connections 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea0c106510 0x7fea0c1a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c105b50 0x7fea0c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe9f00a3bb0 0x7fe9f00a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 --2- 172.21.15.97:0/319683325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0c104960 0x7fea0c19aa30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 >> 172.21.15.97:0/319683325 conn(0x7fea0c0ffeb0 msgr2=0x7fea0c108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 shutdown_connections 2024-04-26T21:30:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.380+0000 7fea11400640 1 -- 172.21.15.97:0/319683325 wait complete. 2024-04-26T21:30:54.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:30:54.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 -- 172.21.15.97:0/3669619815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474106740 msgr2=0x7fd47410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 --2- 172.21.15.97:0/3669619815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474106740 0x7fd47410cfd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009900 tx=0x7fd46c02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 -- 172.21.15.97:0/3669619815 shutdown_connections 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 --2- 172.21.15.97:0/3669619815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474106740 0x7fd47410cfd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 --2- 172.21.15.97:0/3669619815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474105d80 0x7fd474106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 --2- 172.21.15.97:0/3669619815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474104b90 0x7fd474104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.596+0000 7fd47ac00640 1 -- 172.21.15.97:0/3669619815 >> 172.21.15.97:0/3669619815 conn(0x7fd474100120 msgr2=0x7fd474102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.597+0000 7fd47ac00640 1 -- 172.21.15.97:0/3669619815 shutdown_connections 2024-04-26T21:30:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.597+0000 7fd47ac00640 1 -- 172.21.15.97:0/3669619815 wait complete. 2024-04-26T21:30:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 Processor -- start 2024-04-26T21:30:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 -- start start 2024-04-26T21:30:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 0x7fd47419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474106740 0x7fd4741a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741105c0 con 0x7fd474104b90 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd474110440 con 0x7fd474106740 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd47ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd474110740 con 0x7fd474105d80 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd473e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd473e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57896/0 (socket says 172.21.15.97:57896) 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd473e00640 1 -- 172.21.15.97:0/2866235016 learned_addr learned my addr 172.21.15.97:0/2866235016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd473400640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 0x7fd47419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.598+0000 7fd478c00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474106740 0x7fd4741a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 -- 172.21.15.97:0/2866235016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474106740 msgr2=0x7fd4741a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474106740 0x7fd4741a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 -- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 msgr2=0x7fd47419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 -- 172.21.15.97:0/2866235016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4741a2890 con 0x7fd474105d80 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473e00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.599+0000 7fd473400640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 0x7fd47419b1c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.600+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c009c30 con 0x7fd474105d80 2024-04-26T21:30:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.600+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a2b80 con 0x7fd474105d80 2024-04-26T21:30:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.600+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd45c0044c0 con 0x7fd474105d80 2024-04-26T21:30:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.600+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c003d70 con 0x7fd474105d80 2024-04-26T21:30:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.600+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a3120 con 0x7fd474105d80 2024-04-26T21:30:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.601+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47410bf80 con 0x7fd474105d80 2024-04-26T21:30:54.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.604+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd45c003f10 con 0x7fd474105d80 2024-04-26T21:30:54.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.605+0000 7fd470c00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 0x7fd4580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:54.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.605+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd45c0cfda0 con 0x7fd474105d80 2024-04-26T21:30:54.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.605+0000 7fd473e00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 0x7fd4580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:54.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.606+0000 7fd473e00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 0x7fd4580a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd47419c150 tx=0x7fd464009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.608+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd45c017020 con 0x7fd474105d80 2024-04-26T21:30:54.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.821+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd47402dc90 con 0x7fd474105d80 2024-04-26T21:30:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.824+0000 7fd470c00640 1 -- 172.21.15.97:0/2866235016 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22220 (secure 0 0 0) 0x7fd45c09c260 con 0x7fd474105d80 2024-04-26T21:30:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:30:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:30:54.788801+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:30:54.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 msgr2=0x7fd4580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 0x7fd4580a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd47419c150 tx=0x7fd464009500 comp rx=0 tx=0).stop 2024-04-26T21:30:54.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 msgr2=0x7fd47419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 0x7fd47419b1c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c00dde0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 shutdown_connections 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd474106740 0x7fd4741a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd474105d80 0x7fd47419b1c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd4580a3c30 0x7fd4580a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 --2- 172.21.15.97:0/2866235016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd474104b90 0x7fd47419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 >> 172.21.15.97:0/2866235016 conn(0x7fd474100120 msgr2=0x7fd474109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 shutdown_connections 2024-04-26T21:30:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:54.827+0000 7fd47ac00640 1 -- 172.21.15.97:0/2866235016 wait complete. 2024-04-26T21:30:54.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:30:55.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 -- 172.21.15.97:0/3268024561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15000772f0 msgr2=0x7f1500077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 --2- 172.21.15.97:0/3268024561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15000772f0 0x7f1500077710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009900 tx=0x7f14f402a410 comp rx=0 tx=0).stop 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 -- 172.21.15.97:0/3268024561 shutdown_connections 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 --2- 172.21.15.97:0/3268024561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f1500079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 --2- 172.21.15.97:0/3268024561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1500078530 0x7f1500078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 --2- 172.21.15.97:0/3268024561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15000772f0 0x7f1500077710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 -- 172.21.15.97:0/3268024561 >> 172.21.15.97:0/3268024561 conn(0x7f1500100120 msgr2=0x7f1500102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.029+0000 7f1507c00640 1 -- 172.21.15.97:0/3268024561 shutdown_connections 2024-04-26T21:30:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.030+0000 7f1507c00640 1 -- 172.21.15.97:0/3268024561 wait complete. 2024-04-26T21:30:55.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 Processor -- start 2024-04-26T21:30:55.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 -- start start 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15000772f0 0x7f150019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 0x7f150019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.031+0000 7f1507c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f15001149e0 con 0x7f1500078530 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1507c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1500114860 con 0x7f15000772f0 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1507c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1500114b60 con 0x7f1500078ed0 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.97:44726/0 (socket says 172.21.15.97:44726) 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 -- 172.21.15.97:0/1669051154 learned_addr learned my addr 172.21.15.97:0/1669051154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f14ffe00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 0x7f150019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505200640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15000772f0 0x7f150019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 -- 172.21.15.97:0/1669051154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15000772f0 msgr2=0x7f150019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15000772f0 0x7f150019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 -- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 msgr2=0x7f150019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 0x7f150019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f1505c00640 1 -- 172.21.15.97:0/1669051154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15001a6d10 con 0x7f1500078ed0 2024-04-26T21:30:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.032+0000 7f14ffe00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 0x7f150019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f1505c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14f800eb10 tx=0x7f14f800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8004b20 con 0x7f1500078ed0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f8004cc0 con 0x7f1500078ed0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15001a6fa0 con 0x7f1500078ed0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8010690 con 0x7f1500078ed0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.033+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001a74c0 con 0x7f1500078ed0 2024-04-26T21:30:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.034+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1500111520 con 0x7f1500078ed0 2024-04-26T21:30:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.036+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14f8005020 con 0x7f1500078ed0 2024-04-26T21:30:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.038+0000 7f14fd600640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 0x7f14e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.038+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f14f80d0840 con 0x7f1500078ed0 2024-04-26T21:30:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.038+0000 7f1505200640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 0x7f14e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.038+0000 7f1505200640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 0x7f14e80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005ab0 tx=0x7f14f4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.041+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f14f809d2d0 con 0x7f1500078ed0 2024-04-26T21:30:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:54 smithi097 ceph-mon[28127]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:30:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:54 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:30:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:54 smithi097 ceph-mon[28127]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:30:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:54 smithi097 ceph-mon[28127]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:30:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:54 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/319683325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:54 smithi111 ceph-mon[31001]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:30:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:54 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:30:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:54 smithi111 ceph-mon[31001]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:30:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:54 smithi111 ceph-mon[31001]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:30:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:54 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/319683325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:54 smithi143 ceph-mon[31461]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:30:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:54 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3553714969' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:30:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:54 smithi143 ceph-mon[31461]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:30:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:54 smithi143 ceph-mon[31461]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:30:55.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:54 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/319683325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:55.326+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f150002dad0 con 0x7f1500078ed0 2024-04-26T21:30:56.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: pgmap v364: 97 pgs: 41 creating+peering, 6 active+clean, 50 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2866235016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: Cluster is now healthy 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:active 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1669051154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:56.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:56.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:55 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: pgmap v364: 97 pgs: 41 creating+peering, 6 active+clean, 50 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2866235016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: Cluster is now healthy 2024-04-26T21:30:56.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:active 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1669051154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:56.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:55 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.257+0000 7f14fd600640 1 -- 172.21.15.97:0/1669051154 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f14f809cbd0 con 0x7f1500078ed0 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 msgr2=0x7f14e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 0x7f14e80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005ab0 tx=0x7f14f4005a20 comp rx=0 tx=0).stop 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 msgr2=0x7f15001a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14f800eb10 tx=0x7f14f800c2d0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 shutdown_connections 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1500078ed0 0x7f15001a66a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f14e80a3bb0 0x7f14e80a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1500078530 0x7f150019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 --2- 172.21.15.97:0/1669051154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15000772f0 0x7f150019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 >> 172.21.15.97:0/1669051154 conn(0x7f1500100120 msgr2=0x7f1500103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 shutdown_connections 2024-04-26T21:30:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.261+0000 7f1507c00640 1 -- 172.21.15.97:0/1669051154 wait complete. 2024-04-26T21:30:56.286 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T21:30:56.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: pgmap v364: 97 pgs: 41 creating+peering, 6 active+clean, 50 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2866235016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: Cluster is now healthy 2024-04-26T21:30:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] up:active 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1669051154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:56.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:55 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 -- 172.21.15.97:0/1596046987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc101820 msgr2=0x7f26fc101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 --2- 172.21.15.97:0/1596046987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc101820 0x7f26fc101c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009900 tx=0x7f26e802a410 comp rx=0 tx=0).stop 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 -- 172.21.15.97:0/1596046987 shutdown_connections 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 --2- 172.21.15.97:0/1596046987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1021c0 0x7f26fc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 --2- 172.21.15.97:0/1596046987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc101820 0x7f26fc101c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 --2- 172.21.15.97:0/1596046987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc1007b0 0x7f26fc100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 -- 172.21.15.97:0/1596046987 >> 172.21.15.97:0/1596046987 conn(0x7f26fc0fbff0 msgr2=0x7f26fc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 -- 172.21.15.97:0/1596046987 shutdown_connections 2024-04-26T21:30:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.493+0000 7f2701800640 1 -- 172.21.15.97:0/1596046987 wait complete. 2024-04-26T21:30:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.494+0000 7f2701800640 1 Processor -- start 2024-04-26T21:30:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.494+0000 7f2701800640 1 -- start start 2024-04-26T21:30:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1007b0 0x7f26fc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc101820 0x7f26fc19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc110480 con 0x7f26fc1021c0 2024-04-26T21:30:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc110300 con 0x7f26fc101820 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2701800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc110600 con 0x7f26fc1007b0 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57944/0 (socket says 172.21.15.97:57944) 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 -- 172.21.15.97:0/107587018 learned_addr learned my addr 172.21.15.97:0/107587018 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f26fb400640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc101820 0x7f26fc19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f26fbe00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1007b0 0x7f26fc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 -- 172.21.15.97:0/107587018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1007b0 msgr2=0x7f26fc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1007b0 0x7f26fc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 -- 172.21.15.97:0/107587018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc101820 msgr2=0x7f26fc19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc101820 0x7f26fc19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.495+0000 7f2700e00640 1 -- 172.21.15.97:0/107587018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc1a2780 con 0x7f26fc1021c0 2024-04-26T21:30:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f2700e00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c970 tx=0x7f26f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4007bb0 con 0x7f26fc1021c0 2024-04-26T21:30:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26f4007d50 con 0x7f26fc1021c0 2024-04-26T21:30:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc1a2a70 con 0x7f26fc1021c0 2024-04-26T21:30:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004d50 con 0x7f26fc1021c0 2024-04-26T21:30:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.496+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a2fe0 con 0x7f26fc1021c0 2024-04-26T21:30:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.497+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc10ccd0 con 0x7f26fc1021c0 2024-04-26T21:30:56.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.499+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26f4005020 con 0x7f26fc1021c0 2024-04-26T21:30:56.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.501+0000 7f26f8c00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 0x7f26e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:56.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.501+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f26f40d0330 con 0x7f26fc1021c0 2024-04-26T21:30:56.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.501+0000 7f26fbe00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 0x7f26e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:56.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.502+0000 7f26fbe00640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 0x7f26e00a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007350 tx=0x7f26ec0072e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.505+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26f4014030 con 0x7f26fc1021c0 2024-04-26T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:56.749+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f26fc063530 con 0x7f26fc1021c0 2024-04-26T21:30:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.261+0000 7f26f8c00640 1 -- 172.21.15.97:0/107587018 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f26f409cdc0 con 0x7f26fc1021c0 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.265+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 msgr2=0x7f26e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.265+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 0x7f26e00a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007350 tx=0x7f26ec0072e0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.265+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 msgr2=0x7f26fc1a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.265+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26f400c970 tx=0x7f26f400ce30 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 shutdown_connections 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26e00a3c30 0x7f26e00a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26fc1021c0 0x7f26fc1a2110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc101820 0x7f26fc19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 --2- 172.21.15.97:0/107587018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26fc1007b0 0x7f26fc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 >> 172.21.15.97:0/107587018 conn(0x7f26fc0fbff0 msgr2=0x7f26fc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 shutdown_connections 2024-04-26T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.266+0000 7f2701800640 1 -- 172.21.15.97:0/107587018 wait complete. 2024-04-26T21:30:57.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.490+0000 7f3f4be00640 1 -- 172.21.15.97:0/2349701824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101120 msgr2=0x7f3f4c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.490+0000 7f3f4be00640 1 --2- 172.21.15.97:0/2349701824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101120 0x7f3f4c101540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009900 tx=0x7f3f3c02a410 comp rx=0 tx=0).stop 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.490+0000 7f3f4be00640 1 -- 172.21.15.97:0/2349701824 shutdown_connections 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.490+0000 7f3f4be00640 1 --2- 172.21.15.97:0/2349701824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 0x7f3f4c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.490+0000 7f3f4be00640 1 --2- 172.21.15.97:0/2349701824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101a80 0x7f3f4c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.491+0000 7f3f4be00640 1 --2- 172.21.15.97:0/2349701824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101120 0x7f3f4c101540 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.491+0000 7f3f4be00640 1 -- 172.21.15.97:0/2349701824 >> 172.21.15.97:0/2349701824 conn(0x7f3f4c0fcce0 msgr2=0x7f3f4c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.491+0000 7f3f4be00640 1 -- 172.21.15.97:0/2349701824 shutdown_connections 2024-04-26T21:30:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.491+0000 7f3f4be00640 1 -- 172.21.15.97:0/2349701824 wait complete. 2024-04-26T21:30:57.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 Processor -- start 2024-04-26T21:30:57.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 -- start start 2024-04-26T21:30:57.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101120 0x7f3f4c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 0x7f3f4c1a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1103f0 con 0x7f3f4c101a80 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c110270 con 0x7f3f4c101120 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c110570 con 0x7f3f4c102420 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57964/0 (socket says 172.21.15.97:57964) 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.492+0000 7f3f4a000640 1 -- 172.21.15.97:0/1968197436 learned_addr learned my addr 172.21.15.97:0/1968197436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4aa00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101120 0x7f3f4c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4b400640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 0x7f3f4c1a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 -- 172.21.15.97:0/1968197436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 msgr2=0x7f3f4c1a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 0x7f3f4c1a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 -- 172.21.15.97:0/1968197436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101120 msgr2=0x7f3f4c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101120 0x7f3f4c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:57.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 -- 172.21.15.97:0/1968197436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c1a2790 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4a000640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d950 tx=0x7f3f3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f38009c30 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c1a2a80 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800f620 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.493+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c1a2fd0 con 0x7f3f4c101a80 2024-04-26T21:30:57.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.495+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c10c120 con 0x7f3f4c101a80 2024-04-26T21:30:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.497+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3f38010460 con 0x7f3f4c101a80 2024-04-26T21:30:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.499+0000 7f3f43400640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 0x7f3f300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.499+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3f380cfff0 con 0x7f3f4c101a80 2024-04-26T21:30:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.499+0000 7f3f4aa00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 0x7f3f300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.500+0000 7f3f4aa00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 0x7f3f300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.503+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f3809caa0 con 0x7f3f4c101a80 2024-04-26T21:30:57.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:57 smithi143 ceph-mon[31461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:30:57.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:57 smithi143 ceph-mon[31461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:57.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:57 smithi143 ceph-mon[31461]: pgmap v366: 97 pgs: 41 creating+peering, 34 active+clean, 22 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 837 B/s wr, 2 op/s 2024-04-26T21:30:57.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:57 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:30:57.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:57 smithi097 ceph-mon[28127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:30:57.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:57 smithi097 ceph-mon[28127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:57.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:57 smithi097 ceph-mon[28127]: pgmap v366: 97 pgs: 41 creating+peering, 34 active+clean, 22 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 837 B/s wr, 2 op/s 2024-04-26T21:30:57.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:57 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:30:57.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:57 smithi111 ceph-mon[31001]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:30:57.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:57 smithi111 ceph-mon[31001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:57.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:57 smithi111 ceph-mon[31001]: pgmap v366: 97 pgs: 41 creating+peering, 34 active+clean, 22 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 837 B/s wr, 2 op/s 2024-04-26T21:30:57.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:57 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:30:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:57.745+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3f4c063530 con 0x7f3f4c101a80 2024-04-26T21:30:58.142 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:30:57 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: 2024-04-26T21:30:57.736+0000 7fc64e200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:30:58.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.278+0000 7f3f43400640 1 -- 172.21.15.97:0/1968197436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f3f38009de0 con 0x7f3f4c101a80 2024-04-26T21:30:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 msgr2=0x7f3f300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 0x7f3f300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c031400 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 msgr2=0x7f3f4c19b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d950 tx=0x7f3f3800de10 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 shutdown_connections 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f4c102420 0x7f3f4c1a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3f300a3bb0 0x7f3f300a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f4c101a80 0x7f3f4c19b0b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.282+0000 7f3f4be00640 1 --2- 172.21.15.97:0/1968197436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f4c101120 0x7f3f4c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.283+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 >> 172.21.15.97:0/1968197436 conn(0x7f3f4c0fcce0 msgr2=0x7f3f4c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.283+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 shutdown_connections 2024-04-26T21:30:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.283+0000 7f3f4be00640 1 -- 172.21.15.97:0/1968197436 wait complete. 2024-04-26T21:30:58.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 -- 172.21.15.97:0/4219595836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 msgr2=0x7ff2a810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 --2- 172.21.15.97:0/4219595836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a810cfd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009ca0 tx=0x7ff2a002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 -- 172.21.15.97:0/4219595836 shutdown_connections 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 --2- 172.21.15.97:0/4219595836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a810cfd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 --2- 172.21.15.97:0/4219595836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 0x7ff2a8106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 --2- 172.21.15.97:0/4219595836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 0x7ff2a8104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 -- 172.21.15.97:0/4219595836 >> 172.21.15.97:0/4219595836 conn(0x7ff2a81000e0 msgr2=0x7ff2a8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:58.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 -- 172.21.15.97:0/4219595836 shutdown_connections 2024-04-26T21:30:58.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.496+0000 7ff2ade00640 1 -- 172.21.15.97:0/4219595836 wait complete. 2024-04-26T21:30:58.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 Processor -- start 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 -- start start 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 0x7ff2a819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 0x7ff2a819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a81105c0 con 0x7ff2a8106740 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8110440 con 0x7ff2a8104b90 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.497+0000 7ff2ade00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8110740 con 0x7ff2a8105d80 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57978/0 (socket says 172.21.15.97:57978) 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 -- 172.21.15.97:0/2370640341 learned_addr learned my addr 172.21.15.97:0/2370640341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a6a00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 0x7ff2a819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a6000640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 0x7ff2a819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 -- 172.21.15.97:0/2370640341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 msgr2=0x7ff2a819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 0x7ff2a819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 -- 172.21.15.97:0/2370640341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 msgr2=0x7ff2a819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:58.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 0x7ff2a819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 -- 172.21.15.97:0/2370640341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a81a28b0 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.498+0000 7ff2a7400640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002aeb0 tx=0x7ff2a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.499+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003e10 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.499+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2a00044d0 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.499+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0030c90 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.499+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a81a2b40 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.499+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a81a3040 con 0x7ff2a8106740 2024-04-26T21:30:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.500+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a810beb0 con 0x7ff2a8106740 2024-04-26T21:30:58.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.502+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2a0004680 con 0x7ff2a8106740 2024-04-26T21:30:58.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.503+0000 7ff2aca00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 0x7ff28c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:58.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.504+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff2a00f1840 con 0x7ff2a8106740 2024-04-26T21:30:58.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.504+0000 7ff2a6a00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 0x7ff28c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:58.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.504+0000 7ff2a6a00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 0x7ff28c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2a819bb40 tx=0x7ff29800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:58.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.509+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2a00be2d0 con 0x7ff2a8106740 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:30:58.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:58 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:30:58.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:58 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:30:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:30:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/107587018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:30:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:30:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:30:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:30:58.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:30:58.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:58 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:30:58.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:58.751+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff2a802dcf0 con 0x7ff2a8106740 2024-04-26T21:30:59.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.292+0000 7ff2aca00640 1 -- 172.21.15.97:0/2370640341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7ff2a00bdbd0 con 0x7ff2a8106740 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 msgr2=0x7ff28c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 0x7ff28c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2a819bb40 tx=0x7ff29800b410 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 msgr2=0x7ff2a81a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002aeb0 tx=0x7ff2a0003c20 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 shutdown_connections 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff28c0a3c30 0x7ff28c0a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a8106740 0x7ff2a81a2210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2a8105d80 0x7ff2a819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 --2- 172.21.15.97:0/2370640341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2a8104b90 0x7ff2a819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 >> 172.21.15.97:0/2370640341 conn(0x7ff2a81000e0 msgr2=0x7ff2a8109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:59.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 shutdown_connections 2024-04-26T21:30:59.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.296+0000 7ff2ade00640 1 -- 172.21.15.97:0/2370640341 wait complete. 2024-04-26T21:30:59.327 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T21:30:59.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 -- 172.21.15.97:0/3871571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078ed0 msgr2=0x7fde58079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 --2- 172.21.15.97:0/3871571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078ed0 0x7fde58079770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fde50009ca0 tx=0x7fde5002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 -- 172.21.15.97:0/3871571955 shutdown_connections 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 --2- 172.21.15.97:0/3871571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078ed0 0x7fde58079770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 --2- 172.21.15.97:0/3871571955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde58078530 0x7fde58078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 --2- 172.21.15.97:0/3871571955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde580772f0 0x7fde58077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 -- 172.21.15.97:0/3871571955 >> 172.21.15.97:0/3871571955 conn(0x7fde580fffe0 msgr2=0x7fde58102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:30:59.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 -- 172.21.15.97:0/3871571955 shutdown_connections 2024-04-26T21:30:59.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.519+0000 7fde5d200640 1 -- 172.21.15.97:0/3871571955 wait complete. 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 Processor -- start 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 -- start start 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde580772f0 0x7fde5819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde58078ed0 0x7fde581a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde581148a0 con 0x7fde58078530 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fde58114720 con 0x7fde580772f0 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde5d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fde58114a20 con 0x7fde58078ed0 2024-04-26T21:30:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58004/0 (socket says 172.21.15.97:58004) 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.520+0000 7fde4fe00640 1 -- 172.21.15.97:0/268327769 learned_addr learned my addr 172.21.15.97:0/268327769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde57400640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde580772f0 0x7fde5819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde57e00640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde58078ed0 0x7fde581a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 -- 172.21.15.97:0/268327769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde58078ed0 msgr2=0x7fde581a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde58078ed0 0x7fde581a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 -- 172.21.15.97:0/268327769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde580772f0 msgr2=0x7fde5819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde580772f0 0x7fde5819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 -- 172.21.15.97:0/268327769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde581a6ba0 con 0x7fde58078530 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde4fe00640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d950 tx=0x7fde4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40017070 con 0x7fde58078530 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde40009c30 con 0x7fde58078530 2024-04-26T21:30:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4000f620 con 0x7fde58078530 2024-04-26T21:30:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.521+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde581a6e90 con 0x7fde58078530 2024-04-26T21:30:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.522+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581a7350 con 0x7fde58078530 2024-04-26T21:30:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.523+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde581113e0 con 0x7fde58078530 2024-04-26T21:30:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.525+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde40010460 con 0x7fde58078530 2024-04-26T21:30:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.526+0000 7fde54c00640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 0x7fde3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:30:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.526+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fde400cffb0 con 0x7fde58078530 2024-04-26T21:30:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.526+0000 7fde57400640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 0x7fde3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:30:59.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.527+0000 7fde57400640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 0x7fde3c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde480058c0 tx=0x7fde48005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:30:59.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.530+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fde4009cae0 con 0x7fde58078530 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:active 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: pgmap v367: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-26T21:30:59.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:30:59 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:active 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: pgmap v367: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-26T21:30:59.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1968197436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] up:active 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: pgmap v367: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-26T21:30:59.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:30:59 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:30:59.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:30:59.745+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fde58062d70 con 0x7fde3c0a3bb0 2024-04-26T21:31:00.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:30:59.755+0000 7f3a6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:00.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:30:59.755+0000 7f3a6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:00.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:30:59.755+0000 7f3a6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:00.064 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:30:59.755+0000 7f3a6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:00.065 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:30:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:30:59.755+0000 7f3a6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.369+0000 7fde54c00640 1 -- 172.21.15.97:0/268327769 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde58062d70 con 0x7fde3c0a3bb0 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 msgr2=0x7fde3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 0x7fde3c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde480058c0 tx=0x7fde48005c60 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 msgr2=0x7fde5819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d950 tx=0x7fde4000de10 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 shutdown_connections 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde58078ed0 0x7fde581a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fde3c0a3bb0 0x7fde3c0a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde58078530 0x7fde5819f4c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 --2- 172.21.15.97:0/268327769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde580772f0 0x7fde5819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 >> 172.21.15.97:0/268327769 conn(0x7fde580fffe0 msgr2=0x7fde58101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 shutdown_connections 2024-04-26T21:31:00.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.373+0000 7fde5d200640 1 -- 172.21.15.97:0/268327769 wait complete. 2024-04-26T21:31:00.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:active 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:31:00.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:00 smithi097 ceph-mon[28127]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 -- 172.21.15.97:0/2406957732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 msgr2=0x7fd22c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 --2- 172.21.15.97:0/2406957732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c10ccc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd224009ca0 tx=0x7fd22402a950 comp rx=0 tx=0).stop 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 -- 172.21.15.97:0/2406957732 shutdown_connections 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 --2- 172.21.15.97:0/2406957732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c10ccc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 --2- 172.21.15.97:0/2406957732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c075310 0x7fd22c075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 --2- 172.21.15.97:0/2406957732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c0768f0 0x7fd22c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 -- 172.21.15.97:0/2406957732 >> 172.21.15.97:0/2406957732 conn(0x7fd22c0fdf10 msgr2=0x7fd22c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 -- 172.21.15.97:0/2406957732 shutdown_connections 2024-04-26T21:31:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.599+0000 7fd232800640 1 -- 172.21.15.97:0/2406957732 wait complete. 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 Processor -- start 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 -- start start 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 0x7fd22c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c0768f0 0x7fd22c19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1105c0 con 0x7fd22c10a830 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c110440 con 0x7fd22c075310 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd232800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c110740 con 0x7fd22c0768f0 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.600+0000 7fd22be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58040/0 (socket says 172.21.15.97:58040) 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 -- 172.21.15.97:0/566090884 learned_addr learned my addr 172.21.15.97:0/566090884 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22aa00640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c0768f0 0x7fd22c19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22b400640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 0x7fd22c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 -- 172.21.15.97:0/566090884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c0768f0 msgr2=0x7fd22c19b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c0768f0 0x7fd22c19b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 -- 172.21.15.97:0/566090884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 msgr2=0x7fd22c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 0x7fd22c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 -- 172.21.15.97:0/566090884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c1a2900 con 0x7fd22c10a830 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22b400640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 0x7fd22c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd22be00640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd224005e30 tx=0x7fd224003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.601+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224034040 con 0x7fd22c10a830 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.602+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd224030850 con 0x7fd22c10a830 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.602+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c1a2b90 con 0x7fd22c10a830 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.602+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224039430 con 0x7fd22c10a830 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.602+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c1a30d0 con 0x7fd22c10a830 2024-04-26T21:31:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.603+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c1095f0 con 0x7fd22c10a830 2024-04-26T21:31:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.605+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd224037020 con 0x7fd22c10a830 2024-04-26T21:31:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.607+0000 7fd223e00640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 0x7fd2100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.608+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd2240f13e0 con 0x7fd22c10a830 2024-04-26T21:31:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.608+0000 7fd22b400640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 0x7fd2100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.609+0000 7fd22b400640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 0x7fd2100a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd22c19bb90 tx=0x7fd21c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.609+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd2240bde70 con 0x7fd22c10a830 2024-04-26T21:31:00.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:31:00.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:active 2024-04-26T21:31:00.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:31:00.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:31:00.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:31:00.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:31:00.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:00 smithi111 ceph-mon[31001]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2370640341' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] up:active 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:31:00.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:00 smithi143 ceph-mon[31461]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:00.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.845+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd22c062d70 con 0x7fd2100a3c30 2024-04-26T21:31:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.914+0000 7fd223e00640 1 -- 172.21.15.97:0/566090884 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd22c062d70 con 0x7fd2100a3c30 2024-04-26T21:31:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.917+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 msgr2=0x7fd2100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.917+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 0x7fd2100a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd22c19bb90 tx=0x7fd21c005e70 comp rx=0 tx=0).stop 2024-04-26T21:31:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.917+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 msgr2=0x7fd22c1a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.917+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd224005e30 tx=0x7fd224003f10 comp rx=0 tx=0).stop 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 shutdown_connections 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd2100a3c30 0x7fd2100a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd22c10a830 0x7fd22c1a2260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22c0768f0 0x7fd22c19b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 --2- 172.21.15.97:0/566090884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd22c075310 0x7fd22c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 >> 172.21.15.97:0/566090884 conn(0x7fd22c0fdf10 msgr2=0x7fd22c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 shutdown_connections 2024-04-26T21:31:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:00.918+0000 7fd232800640 1 -- 172.21.15.97:0/566090884 wait complete. 2024-04-26T21:31:00.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T21:31:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 -- 172.21.15.97:0/3061795733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 msgr2=0x7f2ce8102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 --2- 172.21.15.97:0/3061795733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce8102630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009900 tx=0x7f2cd002a410 comp rx=0 tx=0).stop 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 -- 172.21.15.97:0/3061795733 shutdown_connections 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 --2- 172.21.15.97:0/3061795733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8106eb0 0x7f2ce81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 --2- 172.21.15.97:0/3061795733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce8102630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 --2- 172.21.15.97:0/3061795733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8068de0 0x7f2ce8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 -- 172.21.15.97:0/3061795733 >> 172.21.15.97:0/3061795733 conn(0x7f2ce80fbff0 msgr2=0x7f2ce80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 -- 172.21.15.97:0/3061795733 shutdown_connections 2024-04-26T21:31:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.145+0000 7f2ceca00640 1 -- 172.21.15.97:0/3061795733 wait complete. 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.146+0000 7f2ceca00640 1 Processor -- start 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 -- start start 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8068de0 0x7f2ce810c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8106eb0 0x7f2ce810b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810e2c0 con 0x7f2ce81021b0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810e140 con 0x7f2ce8106eb0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2ceca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810e440 con 0x7f2ce8068de0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36040/0 (socket says 172.21.15.97:36040) 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.147+0000 7f2cdfe00640 1 -- 172.21.15.97:0/2661071061 learned_addr learned my addr 172.21.15.97:0/2661071061 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ce7400640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8068de0 0x7f2ce810c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ce7e00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8106eb0 0x7f2ce810b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 -- 172.21.15.97:0/2661071061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8068de0 msgr2=0x7f2ce810c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8068de0 0x7f2ce810c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 -- 172.21.15.97:0/2661071061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8106eb0 msgr2=0x7f2ce810b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8106eb0 0x7f2ce810b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 -- 172.21.15.97:0/2661071061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce810ba00 con 0x7f2ce81021b0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2cdfe00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0031460 tx=0x7f2cd0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd003f070 con 0x7f2ce81021b0 2024-04-26T21:31:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cd0031da0 con 0x7f2ce81021b0 2024-04-26T21:31:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce81a2b20 con 0x7f2ce81021b0 2024-04-26T21:31:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.148+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0035030 con 0x7f2ce81021b0 2024-04-26T21:31:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.149+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce81a3070 con 0x7f2ce81021b0 2024-04-26T21:31:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.150+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce8105cc0 con 0x7f2ce81021b0 2024-04-26T21:31:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.152+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2cd0042070 con 0x7f2ce81021b0 2024-04-26T21:31:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.153+0000 7f2ce4c00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 0x7f2cc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.153+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2cd00f0c10 con 0x7f2ce81021b0 2024-04-26T21:31:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.153+0000 7f2ce7400640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 0x7f2cc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.154+0000 7f2ce7400640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 0x7f2cc80a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005fd0 tx=0x7f2cd8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:01.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.158+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2cd00bd6a0 con 0x7f2ce81021b0 2024-04-26T21:31:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.402+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2ce8062e00 con 0x7f2cc80a3bb0 2024-04-26T21:31:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.415+0000 7f2ce4c00640 1 -- 172.21.15.97:0/2661071061 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2ce8062e00 con 0x7f2cc80a3bb0 2024-04-26T21:31:01.416 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_0/a7f60f78-af7e-4156-a7e8-d4c7197c1a4e 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 msgr2=0x7f2cc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 0x7f2cc80a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005fd0 tx=0x7f2cd8005850 comp rx=0 tx=0).stop 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 msgr2=0x7f2ce810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0031460 tx=0x7f2cd0003c80 comp rx=0 tx=0).stop 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 shutdown_connections 2024-04-26T21:31:01.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ce8106eb0 0x7f2ce810b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2cc80a3bb0 0x7f2cc80a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce81021b0 0x7f2ce810ac30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 --2- 172.21.15.97:0/2661071061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ce8068de0 0x7f2ce810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 >> 172.21.15.97:0/2661071061 conn(0x7f2ce80fbff0 msgr2=0x7f2ce8102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 shutdown_connections 2024-04-26T21:31:01.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.418+0000 7f2ceca00640 1 -- 172.21.15.97:0/2661071061 wait complete. 2024-04-26T21:31:01.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:active 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: Cluster is now healthy 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:31:01.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:31:01.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: pgmap v368: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 13 op/s 2024-04-26T21:31:01.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:active 2024-04-26T21:31:01.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:01 smithi097 ceph-mon[28127]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:31:01.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:active 2024-04-26T21:31:01.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:31:01.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:31:01.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: Cluster is now healthy 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: pgmap v368: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 13 op/s 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:active 2024-04-26T21:31:01.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:01 smithi111 ceph-mon[31001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 -- 172.21.15.97:0/1688610842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10a180 msgr2=0x7f970c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 --2- 172.21.15.97:0/1688610842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10a180 0x7f970c10a600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 -- 172.21.15.97:0/1688610842 shutdown_connections 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 --2- 172.21.15.97:0/1688610842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c10ab40 0x7f970c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 --2- 172.21.15.97:0/1688610842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10a180 0x7f970c10a600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 --2- 172.21.15.97:0/1688610842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c108f90 0x7f970c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 -- 172.21.15.97:0/1688610842 >> 172.21.15.97:0/1688610842 conn(0x7f970c077f30 msgr2=0x7f970c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 -- 172.21.15.97:0/1688610842 shutdown_connections 2024-04-26T21:31:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.647+0000 7f9712800640 1 -- 172.21.15.97:0/1688610842 wait complete. 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.648+0000 7f9712800640 1 Processor -- start 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.648+0000 7f9712800640 1 -- start start 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c108f90 0x7f970c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 0x7f970c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f970c114a10 con 0x7f970c10ab40 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f970c114890 con 0x7f970c108f90 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9712800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f970c114b90 con 0x7f970c10a180 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36048/0 (socket says 172.21.15.97:36048) 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 -- 172.21.15.97:0/2141806804 learned_addr learned my addr 172.21.15.97:0/2141806804 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970b400640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c108f90 0x7f970c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9703e00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 0x7f970c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 -- 172.21.15.97:0/2141806804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 msgr2=0x7f970c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 0x7f970c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 -- 172.21.15.97:0/2141806804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c108f90 msgr2=0x7f970c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c108f90 0x7f970c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f970be00640 1 -- 172.21.15.97:0/2141806804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1a6d30 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.649+0000 7f9703e00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 0x7f970c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f970be00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f970400b970 tx=0x7f970400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704019070 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9704014440 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1a6fc0 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97040075d0 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.650+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1a7510 con 0x7f970c10ab40 2024-04-26T21:31:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.651+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c1102d0 con 0x7f970c10ab40 2024-04-26T21:31:01.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.653+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f970401c070 con 0x7f970c10ab40 2024-04-26T21:31:01.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.654+0000 7f9708c00640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 0x7f96f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.655+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f97040cf8b0 con 0x7f970c10ab40 2024-04-26T21:31:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.655+0000 7f970b400640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 0x7f96f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.655+0000 7f970b400640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 0x7f96f00a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0058c0 tx=0x7f96fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:01.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.659+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f970409c3e0 con 0x7f970c10ab40 2024-04-26T21:31:01.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.735+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f97040d1d10 con 0x7f970c10ab40 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] up:active 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: Cluster is now healthy 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:31:01.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: pgmap v368: 97 pgs: 97 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 13 op/s 2024-04-26T21:31:01.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] up:active 2024-04-26T21:31:01.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:01 smithi143 ceph-mon[31461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:31:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.900+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f970c062d70 con 0x7f96f00a3bb0 2024-04-26T21:31:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.970+0000 7f9708c00640 1 -- 172.21.15.97:0/2141806804 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f970c062d70 con 0x7f96f00a3bb0 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 msgr2=0x7f96f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 0x7f96f00a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0058c0 tx=0x7f96fc005850 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 msgr2=0x7f970c1a66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f970400b970 tx=0x7f970400be30 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 shutdown_connections 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f96f00a3bb0 0x7f96f00a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f970c10ab40 0x7f970c1a66c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f970c10a180 0x7f970c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 --2- 172.21.15.97:0/2141806804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f970c108f90 0x7f970c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 >> 172.21.15.97:0/2141806804 conn(0x7f970c077f30 msgr2=0x7f970c10d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 shutdown_connections 2024-04-26T21:31:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:01.973+0000 7f9712800640 1 -- 172.21.15.97:0/2141806804 wait complete. 2024-04-26T21:31:01.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 -- 172.21.15.97:0/612157762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 msgr2=0x7fc4e4077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/612157762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e4077710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009900 tx=0x7fc4d402a410 comp rx=0 tx=0).stop 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 -- 172.21.15.97:0/612157762 shutdown_connections 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/612157762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078ed0 0x7fc4e4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/612157762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078530 0x7fc4e4078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/612157762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e4077710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 -- 172.21.15.97:0/612157762 >> 172.21.15.97:0/612157762 conn(0x7fc4e40fffe0 msgr2=0x7fc4e4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 -- 172.21.15.97:0/612157762 shutdown_connections 2024-04-26T21:31:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.201+0000 7fc4e9a00640 1 -- 172.21.15.97:0/612157762 wait complete. 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 Processor -- start 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 -- start start 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078530 0x7fc4e419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 0x7fc4e41a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41148a0 con 0x7fc4e40772f0 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4114720 con 0x7fc4e4078ed0 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.202+0000 7fc4e9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4114a20 con 0x7fc4e4078530 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36062/0 (socket says 172.21.15.97:36062) 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 -- 172.21.15.97:0/407490550 learned_addr learned my addr 172.21.15.97:0/407490550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4dbe00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078530 0x7fc4e419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e9000640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 0x7fc4e41a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 -- 172.21.15.97:0/407490550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078530 msgr2=0x7fc4e419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078530 0x7fc4e419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 -- 172.21.15.97:0/407490550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 msgr2=0x7fc4e41a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 0x7fc4e41a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 -- 172.21.15.97:0/407490550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e41a6c00 con 0x7fc4e40772f0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e9000640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 0x7fc4e41a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.203+0000 7fc4e3e00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4031460 tx=0x7fc4d4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.204+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d403f070 con 0x7fc4e40772f0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.204+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4d4031da0 con 0x7fc4e40772f0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.204+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4035030 con 0x7fc4e40772f0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.204+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e41a6e90 con 0x7fc4e40772f0 2024-04-26T21:31:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.204+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41a72a0 con 0x7fc4e40772f0 2024-04-26T21:31:02.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.205+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4111610 con 0x7fc4e40772f0 2024-04-26T21:31:02.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.207+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc4d4042070 con 0x7fc4e40772f0 2024-04-26T21:31:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.208+0000 7fc4e1600640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 0x7fc4c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.208+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc4d40f0d00 con 0x7fc4e40772f0 2024-04-26T21:31:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.209+0000 7fc4dbe00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 0x7fc4c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.209+0000 7fc4dbe00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 0x7fc4c80a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.213+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc4d40f4030 con 0x7fc4e40772f0 2024-04-26T21:31:02.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.445+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc4e4062e00 con 0x7fc4c80a3cc0 2024-04-26T21:31:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.459+0000 7fc4e1600640 1 -- 172.21.15.97:0/407490550 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc4e4062e00 con 0x7fc4c80a3cc0 2024-04-26T21:31:02.459 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_1/23120fa4-1db7-404e-8698-301508607e9f 2024-04-26T21:31:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.461+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 msgr2=0x7fc4c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.461+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 0x7fc4c80a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005850 comp rx=0 tx=0).stop 2024-04-26T21:31:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.461+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 msgr2=0x7fc4e419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.461+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4031460 tx=0x7fc4d4003c80 comp rx=0 tx=0).stop 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 shutdown_connections 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4e4078ed0 0x7fc4e41a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4078530 0x7fc4e419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc4c80a3cc0 0x7fc4c80a6180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 --2- 172.21.15.97:0/407490550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e40772f0 0x7fc4e419efe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 >> 172.21.15.97:0/407490550 conn(0x7fc4e40fffe0 msgr2=0x7fc4e4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 shutdown_connections 2024-04-26T21:31:02.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.462+0000 7fc4e9a00640 1 -- 172.21.15.97:0/407490550 wait complete. 2024-04-26T21:31:02.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:31:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 -- 172.21.15.97:0/1554339686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104990 msgr2=0x7fdd20104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 --2- 172.21.15.97:0/1554339686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104990 0x7fdd20104db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009900 tx=0x7fdd1402a410 comp rx=0 tx=0).stop 2024-04-26T21:31:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 -- 172.21.15.97:0/1554339686 shutdown_connections 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 --2- 172.21.15.97:0/1554339686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20106540 0x7fdd2010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 --2- 172.21.15.97:0/1554339686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 0x7fdd20106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 --2- 172.21.15.97:0/1554339686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104990 0x7fdd20104db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 -- 172.21.15.97:0/1554339686 >> 172.21.15.97:0/1554339686 conn(0x7fdd200fff00 msgr2=0x7fdd20102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 -- 172.21.15.97:0/1554339686 shutdown_connections 2024-04-26T21:31:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.665+0000 7fdd26e00640 1 -- 172.21.15.97:0/1554339686 wait complete. 2024-04-26T21:31:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 Processor -- start 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 -- start start 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20104990 0x7fdd2019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 0x7fdd2019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201103c0 con 0x7fdd20106540 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20110240 con 0x7fdd20105b80 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20110540 con 0x7fdd20104990 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36082/0 (socket says 172.21.15.97:36082) 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.667+0000 7fdd26400640 1 -- 172.21.15.97:0/2042943131 learned_addr learned my addr 172.21.15.97:0/2042943131 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd25a00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20104990 0x7fdd2019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd25000640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 0x7fdd2019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 -- 172.21.15.97:0/2042943131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20104990 msgr2=0x7fdd2019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20104990 0x7fdd2019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 -- 172.21.15.97:0/2042943131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 msgr2=0x7fdd2019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 0x7fdd2019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 -- 172.21.15.97:0/2042943131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201a26e0 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26400640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800eaa0 tx=0x7fdd1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1800cdb0 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201a29d0 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.668+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd18005020 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.669+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801f060 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.669+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a2f20 con 0x7fdd20106540 2024-04-26T21:31:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.670+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2010bdd0 con 0x7fdd20106540 2024-04-26T21:31:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.672+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd18004a40 con 0x7fdd20106540 2024-04-26T21:31:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.674+0000 7fdd1e000640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 0x7fdd080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.674+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fdd180d0680 con 0x7fdd20106540 2024-04-26T21:31:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.674+0000 7fdd25a00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 0x7fdd080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.675+0000 7fdd25a00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 0x7fdd080a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd140057c0 tx=0x7fdd14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.679+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdd18016030 con 0x7fdd20106540 2024-04-26T21:31:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:02 smithi097 ceph-mon[28127]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:02 smithi097 ceph-mon[28127]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:02.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:02 smithi097 ceph-mon[28127]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:31:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.923+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd20063530 con 0x7fdd20106540 2024-04-26T21:31:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.924+0000 7fdd1e000640 1 -- 172.21.15.97:0/2042943131 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7fdd1809cfb0 con 0x7fdd20106540 2024-04-26T21:31:02.925 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:02.926 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:31:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:31:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 msgr2=0x7fdd080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 0x7fdd080a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd140057c0 tx=0x7fdd14000bc0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 msgr2=0x7fdd201a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800eaa0 tx=0x7fdd1800ef60 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 shutdown_connections 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fdd080a3c40 0x7fdd080a6100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20106540 0x7fdd201a2070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd20105b80 0x7fdd2019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 --2- 172.21.15.97:0/2042943131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd20104990 0x7fdd2019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.927+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 >> 172.21.15.97:0/2042943131 conn(0x7fdd200fff00 msgr2=0x7fdd20108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.928+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 shutdown_connections 2024-04-26T21:31:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:02.928+0000 7fdd26e00640 1 -- 172.21.15.97:0/2042943131 wait complete. 2024-04-26T21:31:02.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:31:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:02 smithi143 ceph-mon[31461]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:02 smithi143 ceph-mon[31461]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:03.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:02 smithi143 ceph-mon[31461]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:31:03.142 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:31:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: 2024-04-26T21:31:02.737+0000 7fc64e200700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-26T21:31:03.142 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:31:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: 2024-04-26T21:31:02.737+0000 7fc64e200700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-26T21:31:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:02 smithi111 ceph-mon[31001]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:02 smithi111 ceph-mon[31001]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:02 smithi111 ceph-mon[31001]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.146+0000 7f6b2d200640 1 -- 172.21.15.97:0/388383809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b2810a940 msgr2=0x7f6b281111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.146+0000 7f6b2d200640 1 --2- 172.21.15.97:0/388383809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b2810a940 0x7f6b281111d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009ca0 tx=0x7f6b2002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 -- 172.21.15.97:0/388383809 shutdown_connections 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 --2- 172.21.15.97:0/388383809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b2810a940 0x7f6b281111d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 --2- 172.21.15.97:0/388383809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 --2- 172.21.15.97:0/388383809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b28108d90 0x7f6b281091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 -- 172.21.15.97:0/388383809 >> 172.21.15.97:0/388383809 conn(0x7f6b28077d30 msgr2=0x7f6b2807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 -- 172.21.15.97:0/388383809 shutdown_connections 2024-04-26T21:31:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.147+0000 7f6b2d200640 1 -- 172.21.15.97:0/388383809 wait complete. 2024-04-26T21:31:03.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 Processor -- start 2024-04-26T21:31:03.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 -- start start 2024-04-26T21:31:03.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 0x7f6b281a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281147c0 con 0x7f6b28108d90 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28114640 con 0x7f6b28109f80 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b2d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28114940 con 0x7f6b2810a940 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.148+0000 7f6b27400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36100/0 (socket says 172.21.15.97:36100) 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 -- 172.21.15.97:0/3259459844 learned_addr learned my addr 172.21.15.97:0/3259459844 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27e00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 0x7f6b281a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b1ea00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 -- 172.21.15.97:0/3259459844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 msgr2=0x7f6b281a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 0x7f6b281a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 -- 172.21.15.97:0/3259459844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 msgr2=0x7f6b2819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 -- 172.21.15.97:0/3259459844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b281a69f0 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27e00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 0x7f6b281a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b1ea00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.149+0000 7f6b27400640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d950 tx=0x7f6b1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.150+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18017070 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.150+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b18009c30 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.150+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b281a6ce0 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.150+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800f620 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.150+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a7180 con 0x7f6b28108d90 2024-04-26T21:31:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.151+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b281100b0 con 0x7f6b28108d90 2024-04-26T21:31:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.153+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b1801a070 con 0x7f6b28108d90 2024-04-26T21:31:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.154+0000 7f6b24c00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 0x7f6b0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.155+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6b180cffe0 con 0x7f6b28108d90 2024-04-26T21:31:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.155+0000 7f6b1ea00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 0x7f6b0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.155+0000 7f6b1ea00640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 0x7f6b0c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b281a02e0 tx=0x7f6b10005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.159+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b18009de0 con 0x7f6b28108d90 2024-04-26T21:31:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.361+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b28063680 con 0x7f6b28108d90 2024-04-26T21:31:03.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.365+0000 7f6b24c00640 1 -- 172.21.15.97:0/3259459844 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22220 (secure 0 0 0) 0x7f6b1809c930 con 0x7f6b28108d90 2024-04-26T21:31:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:03.367 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:30:54.788801+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 msgr2=0x7f6b0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 0x7f6b0c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b281a02e0 tx=0x7f6b10005d90 comp rx=0 tx=0).stop 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 msgr2=0x7f6b2819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d950 tx=0x7f6b1800de10 comp rx=0 tx=0).stop 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 shutdown_connections 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2810a940 0x7f6b281a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b28109f80 0x7f6b2819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b0c0a3c40 0x7f6b0c0a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 --2- 172.21.15.97:0/3259459844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28108d90 0x7f6b2819edd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 >> 172.21.15.97:0/3259459844 conn(0x7f6b28077d30 msgr2=0x7f6b2810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:03.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 shutdown_connections 2024-04-26T21:31:03.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.368+0000 7f6b2d200640 1 -- 172.21.15.97:0/3259459844 wait complete. 2024-04-26T21:31:03.388 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:31:03.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.583+0000 7f5aeca00640 1 -- 172.21.15.97:0/2069039304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8106540 msgr2=0x7f5ae810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.583+0000 7f5aeca00640 1 --2- 172.21.15.97:0/2069039304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8106540 0x7f5ae810cdd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae002a410 comp rx=0 tx=0).stop 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 -- 172.21.15.97:0/2069039304 shutdown_connections 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 --2- 172.21.15.97:0/2069039304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8106540 0x7f5ae810cdd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 --2- 172.21.15.97:0/2069039304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 0x7f5ae8106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 --2- 172.21.15.97:0/2069039304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8104990 0x7f5ae8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 -- 172.21.15.97:0/2069039304 >> 172.21.15.97:0/2069039304 conn(0x7f5ae80fff00 msgr2=0x7f5ae8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 -- 172.21.15.97:0/2069039304 shutdown_connections 2024-04-26T21:31:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.584+0000 7f5aeca00640 1 -- 172.21.15.97:0/2069039304 wait complete. 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 Processor -- start 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 -- start start 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 0x7f5ae819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 0x7f5ae81a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8110350 con 0x7f5ae8104990 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81101d0 con 0x7f5ae8106540 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.585+0000 7f5aeca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81104d0 con 0x7f5ae8105b80 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36118/0 (socket says 172.21.15.97:36118) 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 -- 172.21.15.97:0/857866820 learned_addr learned my addr 172.21.15.97:0/857866820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5adde00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 0x7f5ae819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae7200640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 0x7f5ae81a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 -- 172.21.15.97:0/857866820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 msgr2=0x7f5ae819af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 0x7f5ae819af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 -- 172.21.15.97:0/857866820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 msgr2=0x7f5ae81a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 0x7f5ae81a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae6800640 1 -- 172.21.15.97:0/857866820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae81a2670 con 0x7f5ae8104990 2024-04-26T21:31:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.586+0000 7f5ae7200640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 0x7f5ae81a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5ae6800640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ea80 tx=0x7f5ad800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800ce40 con 0x7f5ae8104990 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad8011900 con 0x7f5ae8104990 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae81a2960 con 0x7f5ae8104990 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad801a9e0 con 0x7f5ae8104990 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.587+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a2eb0 con 0x7f5ae8104990 2024-04-26T21:31:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.588+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae810bd80 con 0x7f5ae8104990 2024-04-26T21:31:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.590+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ad801ab80 con 0x7f5ae8104990 2024-04-26T21:31:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.591+0000 7f5adfe00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 0x7f5acc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.592+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5ad80d0a70 con 0x7f5ae8104990 2024-04-26T21:31:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.592+0000 7f5adde00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 0x7f5acc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.593+0000 7f5adde00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 0x7f5acc0a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ae819bf60 tx=0x7f5ad000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.596+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ad8016030 con 0x7f5ae8104990 2024-04-26T21:31:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.785+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ae8062d70 con 0x7f5acc0a3c40 2024-04-26T21:31:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.795+0000 7f5adfe00640 1 -- 172.21.15.97:0/857866820 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258853 (secure 0 0 0) 0x7f5ae8062d70 con 0x7f5acc0a3c40 2024-04-26T21:31:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.800+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 msgr2=0x7f5acc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.800+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 0x7f5acc0a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ae819bf60 tx=0x7f5ad000b390 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.800+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 msgr2=0x7f5ae819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.800+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ea80 tx=0x7f5ad800ef40 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 shutdown_connections 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae8106540 0x7f5ae81a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ae8105b80 0x7f5ae819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5acc0a3c40 0x7f5acc0a6100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 --2- 172.21.15.97:0/857866820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae8104990 0x7f5ae819aa50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 >> 172.21.15.97:0/857866820 conn(0x7f5ae80fff00 msgr2=0x7f5ae8108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 shutdown_connections 2024-04-26T21:31:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:03.801+0000 7f5aeca00640 1 -- 172.21.15.97:0/857866820 wait complete. 2024-04-26T21:31:03.821 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-26T21:31:02.654565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616787,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":145,"num_read_kb":133,"num_write":366,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391392,"kb_used_data":13104,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550432,"statfs":{"total":1151940427776,"available":1151539642368,"internally_reserved":0,"allocated":13418496,"data_stored":7230316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26419,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":185,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.864643"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796262+0000","last_change":"2024-04-26T21:30:53.833872+0000","last_active":"2024-04-26T21:30:54.796262+0000","last_peered":"2024-04-26T21:30:54.796262+0000","last_clean":"2024-04-26T21:30:54.796262+0000","last_became_active":"2024-04-26T21:30:53.832109+0000","last_became_peered":"2024-04-26T21:30:53.832109+0000","last_unstale":"2024-04-26T21:30:54.796262+0000","last_undegraded":"2024-04-26T21:30:54.796262+0000","last_fullsized":"2024-04-26T21:30:54.796262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:50.076505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133733+0000","last_change":"2024-04-26T21:30:53.827034+0000","last_active":"2024-04-26T21:30:55.133733+0000","last_peered":"2024-04-26T21:30:55.133733+0000","last_clean":"2024-04-26T21:30:55.133733+0000","last_became_active":"2024-04-26T21:30:53.826682+0000","last_became_peered":"2024-04-26T21:30:53.826682+0000","last_unstale":"2024-04-26T21:30:55.133733+0000","last_undegraded":"2024-04-26T21:30:55.133733+0000","last_fullsized":"2024-04-26T21:30:55.133733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:39.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133748+0000","last_change":"2024-04-26T21:30:53.833196+0000","last_active":"2024-04-26T21:30:55.133748+0000","last_peered":"2024-04-26T21:30:55.133748+0000","last_clean":"2024-04-26T21:30:55.133748+0000","last_became_active":"2024-04-26T21:30:53.831214+0000","last_became_peered":"2024-04-26T21:30:53.831214+0000","last_unstale":"2024-04-26T21:30:55.133748+0000","last_undegraded":"2024-04-26T21:30:55.133748+0000","last_fullsized":"2024-04-26T21:30:55.133748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:17.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133925+0000","last_change":"2024-04-26T21:30:53.829249+0000","last_active":"2024-04-26T21:30:55.133925+0000","last_peered":"2024-04-26T21:30:55.133925+0000","last_clean":"2024-04-26T21:30:55.133925+0000","last_became_active":"2024-04-26T21:30:53.828170+0000","last_became_peered":"2024-04-26T21:30:53.828170+0000","last_unstale":"2024-04-26T21:30:55.133925+0000","last_undegraded":"2024-04-26T21:30:55.133925+0000","last_fullsized":"2024-04-26T21:30:55.133925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:30.729567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965809+0000","last_change":"2024-04-26T21:30:53.824137+0000","last_active":"2024-04-26T21:30:54.965809+0000","last_peered":"2024-04-26T21:30:54.965809+0000","last_clean":"2024-04-26T21:30:54.965809+0000","last_became_active":"2024-04-26T21:30:53.822807+0000","last_became_peered":"2024-04-26T21:30:53.822807+0000","last_unstale":"2024-04-26T21:30:54.965809+0000","last_undegraded":"2024-04-26T21:30:54.965809+0000","last_fullsized":"2024-04-26T21:30:54.965809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:03.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.794932+0000","last_change":"2024-04-26T21:30:53.835932+0000","last_active":"2024-04-26T21:30:54.794932+0000","last_peered":"2024-04-26T21:30:54.794932+0000","last_clean":"2024-04-26T21:30:54.794932+0000","last_became_active":"2024-04-26T21:30:53.835500+0000","last_became_peered":"2024-04-26T21:30:53.835500+0000","last_unstale":"2024-04-26T21:30:54.794932+0000","last_undegraded":"2024-04-26T21:30:54.794932+0000","last_fullsized":"2024-04-26T21:30:54.794932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:34.312999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975197+0000","last_change":"2024-04-26T21:30:53.815175+0000","last_active":"2024-04-26T21:30:54.975197+0000","last_peered":"2024-04-26T21:30:54.975197+0000","last_clean":"2024-04-26T21:30:54.975197+0000","last_became_active":"2024-04-26T21:30:53.814774+0000","last_became_peered":"2024-04-26T21:30:53.814774+0000","last_unstale":"2024-04-26T21:30:54.975197+0000","last_undegraded":"2024-04-26T21:30:54.975197+0000","last_fullsized":"2024-04-26T21:30:54.975197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:21.815872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856133+0000","last_change":"2024-04-26T21:30:53.824718+0000","last_active":"2024-04-26T21:30:54.856133+0000","last_peered":"2024-04-26T21:30:54.856133+0000","last_clean":"2024-04-26T21:30:54.856133+0000","last_became_active":"2024-04-26T21:30:53.824372+0000","last_became_peered":"2024-04-26T21:30:53.824372+0000","last_unstale":"2024-04-26T21:30:54.856133+0000","last_undegraded":"2024-04-26T21:30:54.856133+0000","last_fullsized":"2024-04-26T21:30:54.856133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:33.450906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136604+0000","last_change":"2024-04-26T21:30:53.830237+0000","last_active":"2024-04-26T21:30:55.136604+0000","last_peered":"2024-04-26T21:30:55.136604+0000","last_clean":"2024-04-26T21:30:55.136604+0000","last_became_active":"2024-04-26T21:30:53.829837+0000","last_became_peered":"2024-04-26T21:30:53.829837+0000","last_unstale":"2024-04-26T21:30:55.136604+0000","last_undegraded":"2024-04-26T21:30:55.136604+0000","last_fullsized":"2024-04-26T21:30:55.136604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:33:15.161046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795317+0000","last_change":"2024-04-26T21:30:53.838240+0000","last_active":"2024-04-26T21:30:54.795317+0000","last_peered":"2024-04-26T21:30:54.795317+0000","last_clean":"2024-04-26T21:30:54.795317+0000","last_became_active":"2024-04-26T21:30:53.837741+0000","last_became_peered":"2024-04-26T21:30:53.837741+0000","last_unstale":"2024-04-26T21:30:54.795317+0000","last_undegraded":"2024-04-26T21:30:54.795317+0000","last_fullsized":"2024-04-26T21:30:54.795317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:33.879043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975123+0000","last_change":"2024-04-26T21:30:53.830564+0000","last_active":"2024-04-26T21:30:54.975123+0000","last_peered":"2024-04-26T21:30:54.975123+0000","last_clean":"2024-04-26T21:30:54.975123+0000","last_became_active":"2024-04-26T21:30:53.829967+0000","last_became_peered":"2024-04-26T21:30:53.829967+0000","last_unstale":"2024-04-26T21:30:54.975123+0000","last_undegraded":"2024-04-26T21:30:54.975123+0000","last_fullsized":"2024-04-26T21:30:54.975123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:06.931219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133668+0000","last_change":"2024-04-26T21:30:53.829434+0000","last_active":"2024-04-26T21:30:55.133668+0000","last_peered":"2024-04-26T21:30:55.133668+0000","last_clean":"2024-04-26T21:30:55.133668+0000","last_became_active":"2024-04-26T21:30:53.828657+0000","last_became_peered":"2024-04-26T21:30:53.828657+0000","last_unstale":"2024-04-26T21:30:55.133668+0000","last_undegraded":"2024-04-26T21:30:55.133668+0000","last_fullsized":"2024-04-26T21:30:55.133668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:12.215856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133005+0000","last_change":"2024-04-26T21:30:53.824833+0000","last_active":"2024-04-26T21:30:55.133005+0000","last_peered":"2024-04-26T21:30:55.133005+0000","last_clean":"2024-04-26T21:30:55.133005+0000","last_became_active":"2024-04-26T21:30:53.824489+0000","last_became_peered":"2024-04-26T21:30:53.824489+0000","last_unstale":"2024-04-26T21:30:55.133005+0000","last_undegraded":"2024-04-26T21:30:55.133005+0000","last_fullsized":"2024-04-26T21:30:55.133005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132977+0000","last_change":"2024-04-26T21:30:53.824573+0000","last_active":"2024-04-26T21:30:55.132977+0000","last_peered":"2024-04-26T21:30:55.132977+0000","last_clean":"2024-04-26T21:30:55.132977+0000","last_became_active":"2024-04-26T21:30:53.824242+0000","last_became_peered":"2024-04-26T21:30:53.824242+0000","last_unstale":"2024-04-26T21:30:55.132977+0000","last_undegraded":"2024-04-26T21:30:55.132977+0000","last_fullsized":"2024-04-26T21:30:55.132977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:12.271919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133194+0000","last_change":"2024-04-26T21:30:53.827619+0000","last_active":"2024-04-26T21:30:55.133194+0000","last_peered":"2024-04-26T21:30:55.133194+0000","last_clean":"2024-04-26T21:30:55.133194+0000","last_became_active":"2024-04-26T21:30:53.827301+0000","last_became_peered":"2024-04-26T21:30:53.827301+0000","last_unstale":"2024-04-26T21:30:55.133194+0000","last_undegraded":"2024-04-26T21:30:55.133194+0000","last_fullsized":"2024-04-26T21:30:55.133194+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:25.152302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134868+0000","last_change":"2024-04-26T21:30:53.825785+0000","last_active":"2024-04-26T21:30:55.134868+0000","last_peered":"2024-04-26T21:30:55.134868+0000","last_clean":"2024-04-26T21:30:55.134868+0000","last_became_active":"2024-04-26T21:30:53.821673+0000","last_became_peered":"2024-04-26T21:30:53.821673+0000","last_unstale":"2024-04-26T21:30:55.134868+0000","last_undegraded":"2024-04-26T21:30:55.134868+0000","last_fullsized":"2024-04-26T21:30:55.134868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:39.138995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796284+0000","last_change":"2024-04-26T21:30:53.834768+0000","last_active":"2024-04-26T21:30:54.796284+0000","last_peered":"2024-04-26T21:30:54.796284+0000","last_clean":"2024-04-26T21:30:54.796284+0000","last_became_active":"2024-04-26T21:30:53.831624+0000","last_became_peered":"2024-04-26T21:30:53.831624+0000","last_unstale":"2024-04-26T21:30:54.796284+0000","last_undegraded":"2024-04-26T21:30:54.796284+0000","last_fullsized":"2024-04-26T21:30:54.796284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:49.228987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.797327+0000","last_change":"2024-04-26T21:30:53.828348+0000","last_active":"2024-04-26T21:30:54.797327+0000","last_peered":"2024-04-26T21:30:54.797327+0000","last_clean":"2024-04-26T21:30:54.797327+0000","last_became_active":"2024-04-26T21:30:53.827926+0000","last_became_peered":"2024-04-26T21:30:53.827926+0000","last_unstale":"2024-04-26T21:30:54.797327+0000","last_undegraded":"2024-04-26T21:30:54.797327+0000","last_fullsized":"2024-04-26T21:30:54.797327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133966+0000","last_change":"2024-04-26T21:30:53.824205+0000","last_active":"2024-04-26T21:30:55.133966+0000","last_peered":"2024-04-26T21:30:55.133966+0000","last_clean":"2024-04-26T21:30:55.133966+0000","last_became_active":"2024-04-26T21:30:53.823787+0000","last_became_peered":"2024-04-26T21:30:53.823787+0000","last_unstale":"2024-04-26T21:30:55.133966+0000","last_undegraded":"2024-04-26T21:30:55.133966+0000","last_fullsized":"2024-04-26T21:30:55.133966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:23.325924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'11","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.325361+0000","last_change":"2024-04-26T21:30:52.803835+0000","last_active":"2024-04-26T21:30:59.325361+0000","last_peered":"2024-04-26T21:30:59.325361+0000","last_clean":"2024-04-26T21:30:59.325361+0000","last_became_active":"2024-04-26T21:30:52.803541+0000","last_became_peered":"2024-04-26T21:30:52.803541+0000","last_unstale":"2024-04-26T21:30:59.325361+0000","last_undegraded":"2024-04-26T21:30:59.325361+0000","last_fullsized":"2024-04-26T21:30:59.325361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:16.314776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856814+0000","last_change":"2024-04-26T21:30:53.823811+0000","last_active":"2024-04-26T21:30:54.856814+0000","last_peered":"2024-04-26T21:30:54.856814+0000","last_clean":"2024-04-26T21:30:54.856814+0000","last_became_active":"2024-04-26T21:30:53.823284+0000","last_became_peered":"2024-04-26T21:30:53.823284+0000","last_unstale":"2024-04-26T21:30:54.856814+0000","last_undegraded":"2024-04-26T21:30:54.856814+0000","last_fullsized":"2024-04-26T21:30:54.856814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:05.627445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856825+0000","last_change":"2024-04-26T21:30:52.810994+0000","last_active":"2024-04-26T21:30:54.856825+0000","last_peered":"2024-04-26T21:30:54.856825+0000","last_clean":"2024-04-26T21:30:54.856825+0000","last_became_active":"2024-04-26T21:30:52.809509+0000","last_became_peered":"2024-04-26T21:30:52.809509+0000","last_unstale":"2024-04-26T21:30:54.856825+0000","last_undegraded":"2024-04-26T21:30:54.856825+0000","last_fullsized":"2024-04-26T21:30:54.856825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:24.366289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133952+0000","last_change":"2024-04-26T21:30:53.830471+0000","last_active":"2024-04-26T21:30:55.133952+0000","last_peered":"2024-04-26T21:30:55.133952+0000","last_clean":"2024-04-26T21:30:55.133952+0000","last_became_active":"2024-04-26T21:30:53.829841+0000","last_became_peered":"2024-04-26T21:30:53.829841+0000","last_unstale":"2024-04-26T21:30:55.133952+0000","last_undegraded":"2024-04-26T21:30:55.133952+0000","last_fullsized":"2024-04-26T21:30:55.133952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:25.988137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.705675+0000","last_change":"2024-04-26T21:30:52.812564+0000","last_active":"2024-04-26T21:31:00.705675+0000","last_peered":"2024-04-26T21:31:00.705675+0000","last_clean":"2024-04-26T21:31:00.705675+0000","last_became_active":"2024-04-26T21:30:52.811090+0000","last_became_peered":"2024-04-26T21:30:52.811090+0000","last_unstale":"2024-04-26T21:31:00.705675+0000","last_undegraded":"2024-04-26T21:31:00.705675+0000","last_fullsized":"2024-04-26T21:31:00.705675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:51.522689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.135859+0000","last_change":"2024-04-26T21:30:53.828398+0000","last_active":"2024-04-26T21:30:55.135859+0000","last_peered":"2024-04-26T21:30:55.135859+0000","last_clean":"2024-04-26T21:30:55.135859+0000","last_became_active":"2024-04-26T21:30:53.827920+0000","last_became_peered":"2024-04-26T21:30:53.827920+0000","last_unstale":"2024-04-26T21:30:55.135859+0000","last_undegraded":"2024-04-26T21:30:55.135859+0000","last_fullsized":"2024-04-26T21:30:55.135859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:06.463121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.341294+0000","last_change":"2024-04-26T21:30:52.798297+0000","last_active":"2024-04-26T21:31:00.341294+0000","last_peered":"2024-04-26T21:31:00.341294+0000","last_clean":"2024-04-26T21:31:00.341294+0000","last_became_active":"2024-04-26T21:30:52.797998+0000","last_became_peered":"2024-04-26T21:30:52.797998+0000","last_unstale":"2024-04-26T21:31:00.341294+0000","last_undegraded":"2024-04-26T21:31:00.341294+0000","last_fullsized":"2024-04-26T21:31:00.341294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:40.790353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134028+0000","last_change":"2024-04-26T21:30:53.830846+0000","last_active":"2024-04-26T21:30:55.134028+0000","last_peered":"2024-04-26T21:30:55.134028+0000","last_clean":"2024-04-26T21:30:55.134028+0000","last_became_active":"2024-04-26T21:30:53.830294+0000","last_became_peered":"2024-04-26T21:30:53.830294+0000","last_unstale":"2024-04-26T21:30:55.134028+0000","last_undegraded":"2024-04-26T21:30:55.134028+0000","last_fullsized":"2024-04-26T21:30:55.134028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:50.929819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.301190+0000","last_change":"2024-04-26T21:30:52.807155+0000","last_active":"2024-04-26T21:30:57.301190+0000","last_peered":"2024-04-26T21:30:57.301190+0000","last_clean":"2024-04-26T21:30:57.301190+0000","last_became_active":"2024-04-26T21:30:52.805230+0000","last_became_peered":"2024-04-26T21:30:52.805230+0000","last_unstale":"2024-04-26T21:30:57.301190+0000","last_undegraded":"2024-04-26T21:30:57.301190+0000","last_fullsized":"2024-04-26T21:30:57.301190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:44.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965493+0000","last_change":"2024-04-26T21:30:53.810334+0000","last_active":"2024-04-26T21:30:54.965493+0000","last_peered":"2024-04-26T21:30:54.965493+0000","last_clean":"2024-04-26T21:30:54.965493+0000","last_became_active":"2024-04-26T21:30:53.809867+0000","last_became_peered":"2024-04-26T21:30:53.809867+0000","last_unstale":"2024-04-26T21:30:54.965493+0000","last_undegraded":"2024-04-26T21:30:54.965493+0000","last_fullsized":"2024-04-26T21:30:54.965493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:46.377262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.327103+0000","last_change":"2024-04-26T21:30:52.804775+0000","last_active":"2024-04-26T21:30:59.327103+0000","last_peered":"2024-04-26T21:30:59.327103+0000","last_clean":"2024-04-26T21:30:59.327103+0000","last_became_active":"2024-04-26T21:30:52.804319+0000","last_became_peered":"2024-04-26T21:30:52.804319+0000","last_unstale":"2024-04-26T21:30:59.327103+0000","last_undegraded":"2024-04-26T21:30:59.327103+0000","last_fullsized":"2024-04-26T21:30:59.327103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:12.761080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133866+0000","last_change":"2024-04-26T21:30:53.823021+0000","last_active":"2024-04-26T21:30:55.133866+0000","last_peered":"2024-04-26T21:30:55.133866+0000","last_clean":"2024-04-26T21:30:55.133866+0000","last_became_active":"2024-04-26T21:30:53.822633+0000","last_became_peered":"2024-04-26T21:30:53.822633+0000","last_unstale":"2024-04-26T21:30:55.133866+0000","last_undegraded":"2024-04-26T21:30:55.133866+0000","last_fullsized":"2024-04-26T21:30:55.133866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:32.178472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.314848+0000","last_change":"2024-04-26T21:30:52.796729+0000","last_active":"2024-04-26T21:30:58.314848+0000","last_peered":"2024-04-26T21:30:58.314848+0000","last_clean":"2024-04-26T21:30:58.314848+0000","last_became_active":"2024-04-26T21:30:52.796341+0000","last_became_peered":"2024-04-26T21:30:52.796341+0000","last_unstale":"2024-04-26T21:30:58.314848+0000","last_undegraded":"2024-04-26T21:30:58.314848+0000","last_fullsized":"2024-04-26T21:30:58.314848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:30.434541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.855935+0000","last_change":"2024-04-26T21:30:53.816136+0000","last_active":"2024-04-26T21:30:54.855935+0000","last_peered":"2024-04-26T21:30:54.855935+0000","last_clean":"2024-04-26T21:30:54.855935+0000","last_became_active":"2024-04-26T21:30:53.815758+0000","last_became_peered":"2024-04-26T21:30:53.815758+0000","last_unstale":"2024-04-26T21:30:54.855935+0000","last_undegraded":"2024-04-26T21:30:54.855935+0000","last_fullsized":"2024-04-26T21:30:54.855935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:41.606483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133180+0000","last_change":"2024-04-26T21:30:52.804102+0000","last_active":"2024-04-26T21:30:55.133180+0000","last_peered":"2024-04-26T21:30:55.133180+0000","last_clean":"2024-04-26T21:30:55.133180+0000","last_became_active":"2024-04-26T21:30:52.803808+0000","last_became_peered":"2024-04-26T21:30:52.803808+0000","last_unstale":"2024-04-26T21:30:55.133180+0000","last_undegraded":"2024-04-26T21:30:55.133180+0000","last_fullsized":"2024-04-26T21:30:55.133180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:16.357651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796196+0000","last_change":"2024-04-26T21:30:53.822475+0000","last_active":"2024-04-26T21:30:54.796196+0000","last_peered":"2024-04-26T21:30:54.796196+0000","last_clean":"2024-04-26T21:30:54.796196+0000","last_became_active":"2024-04-26T21:30:53.822077+0000","last_became_peered":"2024-04-26T21:30:53.822077+0000","last_unstale":"2024-04-26T21:30:54.796196+0000","last_undegraded":"2024-04-26T21:30:54.796196+0000","last_fullsized":"2024-04-26T21:30:54.796196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:09.289557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.708579+0000","last_change":"2024-04-26T21:30:52.802080+0000","last_active":"2024-04-26T21:31:00.708579+0000","last_peered":"2024-04-26T21:31:00.708579+0000","last_clean":"2024-04-26T21:31:00.708579+0000","last_became_active":"2024-04-26T21:30:52.801645+0000","last_became_peered":"2024-04-26T21:30:52.801645+0000","last_unstale":"2024-04-26T21:31:00.708579+0000","last_undegraded":"2024-04-26T21:31:00.708579+0000","last_fullsized":"2024-04-26T21:31:00.708579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:00.213924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136085+0000","last_change":"2024-04-26T21:30:53.832936+0000","last_active":"2024-04-26T21:30:55.136085+0000","last_peered":"2024-04-26T21:30:55.136085+0000","last_clean":"2024-04-26T21:30:55.136085+0000","last_became_active":"2024-04-26T21:30:53.830821+0000","last_became_peered":"2024-04-26T21:30:53.830821+0000","last_unstale":"2024-04-26T21:30:55.136085+0000","last_undegraded":"2024-04-26T21:30:55.136085+0000","last_fullsized":"2024-04-26T21:30:55.136085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:08.540653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.300486+0000","last_change":"2024-04-26T21:30:52.812687+0000","last_active":"2024-04-26T21:30:57.300486+0000","last_peered":"2024-04-26T21:30:57.300486+0000","last_clean":"2024-04-26T21:30:57.300486+0000","last_became_active":"2024-04-26T21:30:52.812055+0000","last_became_peered":"2024-04-26T21:30:52.812055+0000","last_unstale":"2024-04-26T21:30:57.300486+0000","last_undegraded":"2024-04-26T21:30:57.300486+0000","last_fullsized":"2024-04-26T21:30:57.300486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:51.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133998+0000","last_change":"2024-04-26T21:30:53.829887+0000","last_active":"2024-04-26T21:30:55.133998+0000","last_peered":"2024-04-26T21:30:55.133998+0000","last_clean":"2024-04-26T21:30:55.133998+0000","last_became_active":"2024-04-26T21:30:53.829355+0000","last_became_peered":"2024-04-26T21:30:53.829355+0000","last_unstale":"2024-04-26T21:30:55.133998+0000","last_undegraded":"2024-04-26T21:30:55.133998+0000","last_fullsized":"2024-04-26T21:30:55.133998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:49.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134085+0000","last_change":"2024-04-26T21:30:52.808859+0000","last_active":"2024-04-26T21:30:55.134085+0000","last_peered":"2024-04-26T21:30:55.134085+0000","last_clean":"2024-04-26T21:30:55.134085+0000","last_became_active":"2024-04-26T21:30:52.807232+0000","last_became_peered":"2024-04-26T21:30:52.807232+0000","last_unstale":"2024-04-26T21:30:55.134085+0000","last_undegraded":"2024-04-26T21:30:55.134085+0000","last_fullsized":"2024-04-26T21:30:55.134085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:44.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136231+0000","last_change":"2024-04-26T21:30:53.829723+0000","last_active":"2024-04-26T21:30:55.136231+0000","last_peered":"2024-04-26T21:30:55.136231+0000","last_clean":"2024-04-26T21:30:55.136231+0000","last_became_active":"2024-04-26T21:30:53.827799+0000","last_became_peered":"2024-04-26T21:30:53.827799+0000","last_unstale":"2024-04-26T21:30:55.136231+0000","last_undegraded":"2024-04-26T21:30:55.136231+0000","last_fullsized":"2024-04-26T21:30:55.136231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:20.798100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'17","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.889467+0000","last_change":"2024-04-26T21:30:52.812217+0000","last_active":"2024-04-26T21:31:00.889467+0000","last_peered":"2024-04-26T21:31:00.889467+0000","last_clean":"2024-04-26T21:31:00.889467+0000","last_became_active":"2024-04-26T21:30:52.811730+0000","last_became_peered":"2024-04-26T21:30:52.811730+0000","last_unstale":"2024-04-26T21:31:00.889467+0000","last_undegraded":"2024-04-26T21:31:00.889467+0000","last_fullsized":"2024-04-26T21:31:00.889467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:08.012161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":10,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965598+0000","last_change":"2024-04-26T21:30:53.813388+0000","last_active":"2024-04-26T21:30:54.965598+0000","last_peered":"2024-04-26T21:30:54.965598+0000","last_clean":"2024-04-26T21:30:54.965598+0000","last_became_active":"2024-04-26T21:30:53.812972+0000","last_became_peered":"2024-04-26T21:30:53.812972+0000","last_unstale":"2024-04-26T21:30:54.965598+0000","last_undegraded":"2024-04-26T21:30:54.965598+0000","last_fullsized":"2024-04-26T21:30:54.965598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:22.480641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.706119+0000","last_change":"2024-04-26T21:30:52.804426+0000","last_active":"2024-04-26T21:31:00.706119+0000","last_peered":"2024-04-26T21:31:00.706119+0000","last_clean":"2024-04-26T21:31:00.706119+0000","last_became_active":"2024-04-26T21:30:52.804083+0000","last_became_peered":"2024-04-26T21:30:52.804083+0000","last_unstale":"2024-04-26T21:31:00.706119+0000","last_undegraded":"2024-04-26T21:31:00.706119+0000","last_fullsized":"2024-04-26T21:31:00.706119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:10.164976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132691+0000","last_change":"2024-04-26T21:30:53.829326+0000","last_active":"2024-04-26T21:30:55.132691+0000","last_peered":"2024-04-26T21:30:55.132691+0000","last_clean":"2024-04-26T21:30:55.132691+0000","last_became_active":"2024-04-26T21:30:53.828863+0000","last_became_peered":"2024-04-26T21:30:53.828863+0000","last_unstale":"2024-04-26T21:30:55.132691+0000","last_undegraded":"2024-04-26T21:30:55.132691+0000","last_fullsized":"2024-04-26T21:30:55.132691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:56.756187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.298219+0000","last_change":"2024-04-26T21:30:52.805302+0000","last_active":"2024-04-26T21:30:57.298219+0000","last_peered":"2024-04-26T21:30:57.298219+0000","last_clean":"2024-04-26T21:30:57.298219+0000","last_became_active":"2024-04-26T21:30:52.804869+0000","last_became_peered":"2024-04-26T21:30:52.804869+0000","last_unstale":"2024-04-26T21:30:57.298219+0000","last_undegraded":"2024-04-26T21:30:57.298219+0000","last_fullsized":"2024-04-26T21:30:57.298219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:21.299474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974910+0000","last_change":"2024-04-26T21:30:53.823827+0000","last_active":"2024-04-26T21:30:54.974910+0000","last_peered":"2024-04-26T21:30:54.974910+0000","last_clean":"2024-04-26T21:30:54.974910+0000","last_became_active":"2024-04-26T21:30:53.823492+0000","last_became_peered":"2024-04-26T21:30:53.823492+0000","last_unstale":"2024-04-26T21:30:54.974910+0000","last_undegraded":"2024-04-26T21:30:54.974910+0000","last_fullsized":"2024-04-26T21:30:54.974910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.514753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.316576+0000","last_change":"2024-04-26T21:30:52.801468+0000","last_active":"2024-04-26T21:30:59.316576+0000","last_peered":"2024-04-26T21:30:59.316576+0000","last_clean":"2024-04-26T21:30:59.316576+0000","last_became_active":"2024-04-26T21:30:52.801071+0000","last_became_peered":"2024-04-26T21:30:52.801071+0000","last_unstale":"2024-04-26T21:30:59.316576+0000","last_undegraded":"2024-04-26T21:30:59.316576+0000","last_fullsized":"2024-04-26T21:30:59.316576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:52.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974518+0000","last_change":"2024-04-26T21:30:53.824075+0000","last_active":"2024-04-26T21:30:54.974518+0000","last_peered":"2024-04-26T21:30:54.974518+0000","last_clean":"2024-04-26T21:30:54.974518+0000","last_became_active":"2024-04-26T21:30:53.823753+0000","last_became_peered":"2024-04-26T21:30:53.823753+0000","last_unstale":"2024-04-26T21:30:54.974518+0000","last_undegraded":"2024-04-26T21:30:54.974518+0000","last_fullsized":"2024-04-26T21:30:54.974518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:44:09.294481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.342685+0000","last_change":"2024-04-26T21:30:52.806725+0000","last_active":"2024-04-26T21:31:00.342685+0000","last_peered":"2024-04-26T21:31:00.342685+0000","last_clean":"2024-04-26T21:31:00.342685+0000","last_became_active":"2024-04-26T21:30:52.806318+0000","last_became_peered":"2024-04-26T21:30:52.806318+0000","last_unstale":"2024-04-26T21:31:00.342685+0000","last_undegraded":"2024-04-26T21:31:00.342685+0000","last_fullsized":"2024-04-26T21:31:00.342685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:28.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133114+0000","last_change":"2024-04-26T21:30:53.835273+0000","last_active":"2024-04-26T21:30:55.133114+0000","last_peered":"2024-04-26T21:30:55.133114+0000","last_clean":"2024-04-26T21:30:55.133114+0000","last_became_active":"2024-04-26T21:30:53.834922+0000","last_became_peered":"2024-04-26T21:30:53.834922+0000","last_unstale":"2024-04-26T21:30:55.133114+0000","last_undegraded":"2024-04-26T21:30:55.133114+0000","last_fullsized":"2024-04-26T21:30:55.133114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:43.992579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.307685+0000","last_change":"2024-04-26T21:30:52.797639+0000","last_active":"2024-04-26T21:30:59.307685+0000","last_peered":"2024-04-26T21:30:59.307685+0000","last_clean":"2024-04-26T21:30:59.307685+0000","last_became_active":"2024-04-26T21:30:52.796576+0000","last_became_peered":"2024-04-26T21:30:52.796576+0000","last_unstale":"2024-04-26T21:30:59.307685+0000","last_undegraded":"2024-04-26T21:30:59.307685+0000","last_fullsized":"2024-04-26T21:30:59.307685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.730232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795853+0000","last_change":"2024-04-26T21:30:53.836190+0000","last_active":"2024-04-26T21:30:54.795853+0000","last_peered":"2024-04-26T21:30:54.795853+0000","last_clean":"2024-04-26T21:30:54.795853+0000","last_became_active":"2024-04-26T21:30:53.835302+0000","last_became_peered":"2024-04-26T21:30:53.835302+0000","last_unstale":"2024-04-26T21:30:54.795853+0000","last_undegraded":"2024-04-26T21:30:54.795853+0000","last_fullsized":"2024-04-26T21:30:54.795853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:13.222578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.206758+0000","last_change":"2024-04-26T21:30:52.809928+0000","last_active":"2024-04-26T21:30:55.206758+0000","last_peered":"2024-04-26T21:30:55.206758+0000","last_clean":"2024-04-26T21:30:55.206758+0000","last_became_active":"2024-04-26T21:30:52.808667+0000","last_became_peered":"2024-04-26T21:30:52.808667+0000","last_unstale":"2024-04-26T21:30:55.206758+0000","last_undegraded":"2024-04-26T21:30:55.206758+0000","last_fullsized":"2024-04-26T21:30:55.206758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:15.375105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:56.175776+0000","last_change":"2024-04-26T21:30:53.836649+0000","last_active":"2024-04-26T21:30:56.175776+0000","last_peered":"2024-04-26T21:30:56.175776+0000","last_clean":"2024-04-26T21:30:56.175776+0000","last_became_active":"2024-04-26T21:30:53.836252+0000","last_became_peered":"2024-04-26T21:30:53.836252+0000","last_unstale":"2024-04-26T21:30:56.175776+0000","last_undegraded":"2024-04-26T21:30:56.175776+0000","last_fullsized":"2024-04-26T21:30:56.175776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:53.797872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.222956+0000","last_change":"2024-04-26T21:30:52.808196+0000","last_active":"2024-04-26T21:30:55.222956+0000","last_peered":"2024-04-26T21:30:55.222956+0000","last_clean":"2024-04-26T21:30:55.222956+0000","last_became_active":"2024-04-26T21:30:52.806445+0000","last_became_peered":"2024-04-26T21:30:52.806445+0000","last_unstale":"2024-04-26T21:30:55.222956+0000","last_undegraded":"2024-04-26T21:30:55.222956+0000","last_fullsized":"2024-04-26T21:30:55.222956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:39.815485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.794891+0000","last_change":"2024-04-26T21:30:53.837998+0000","last_active":"2024-04-26T21:30:54.794891+0000","last_peered":"2024-04-26T21:30:54.794891+0000","last_clean":"2024-04-26T21:30:54.794891+0000","last_became_active":"2024-04-26T21:30:53.837443+0000","last_became_peered":"2024-04-26T21:30:53.837443+0000","last_unstale":"2024-04-26T21:30:54.794891+0000","last_undegraded":"2024-04-26T21:30:54.794891+0000","last_fullsized":"2024-04-26T21:30:54.794891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:04.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133492+0000","last_change":"2024-04-26T21:30:53.835501+0000","last_active":"2024-04-26T21:30:55.133492+0000","last_peered":"2024-04-26T21:30:55.133492+0000","last_clean":"2024-04-26T21:30:55.133492+0000","last_became_active":"2024-04-26T21:30:53.835096+0000","last_became_peered":"2024-04-26T21:30:53.835096+0000","last_unstale":"2024-04-26T21:30:55.133492+0000","last_undegraded":"2024-04-26T21:30:55.133492+0000","last_fullsized":"2024-04-26T21:30:55.133492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:26.178001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.316655+0000","last_change":"2024-04-26T21:30:52.805357+0000","last_active":"2024-04-26T21:30:58.316655+0000","last_peered":"2024-04-26T21:30:58.316655+0000","last_clean":"2024-04-26T21:30:58.316655+0000","last_became_active":"2024-04-26T21:30:52.804902+0000","last_became_peered":"2024-04-26T21:30:52.804902+0000","last_unstale":"2024-04-26T21:30:58.316655+0000","last_undegraded":"2024-04-26T21:30:58.316655+0000","last_fullsized":"2024-04-26T21:30:58.316655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:29.276141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856327+0000","last_change":"2024-04-26T21:30:53.835448+0000","last_active":"2024-04-26T21:30:54.856327+0000","last_peered":"2024-04-26T21:30:54.856327+0000","last_clean":"2024-04-26T21:30:54.856327+0000","last_became_active":"2024-04-26T21:30:53.835054+0000","last_became_peered":"2024-04-26T21:30:53.835054+0000","last_unstale":"2024-04-26T21:30:54.856327+0000","last_undegraded":"2024-04-26T21:30:54.856327+0000","last_fullsized":"2024-04-26T21:30:54.856327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:37.892713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.318992+0000","last_change":"2024-04-26T21:30:52.792942+0000","last_active":"2024-04-26T21:30:59.318992+0000","last_peered":"2024-04-26T21:30:59.318992+0000","last_clean":"2024-04-26T21:30:59.318992+0000","last_became_active":"2024-04-26T21:30:52.792553+0000","last_became_peered":"2024-04-26T21:30:52.792553+0000","last_unstale":"2024-04-26T21:30:59.318992+0000","last_undegraded":"2024-04-26T21:30:59.318992+0000","last_fullsized":"2024-04-26T21:30:59.318992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.915883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974531+0000","last_change":"2024-04-26T21:29:15.217076+0000","last_active":"2024-04-26T21:30:54.974531+0000","last_peered":"2024-04-26T21:30:54.974531+0000","last_clean":"2024-04-26T21:30:54.974531+0000","last_became_active":"2024-04-26T21:29:15.215786+0000","last_became_peered":"2024-04-26T21:29:15.215786+0000","last_unstale":"2024-04-26T21:30:54.974531+0000","last_undegraded":"2024-04-26T21:30:54.974531+0000","last_fullsized":"2024-04-26T21:30:54.974531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_clean_scrub_stamp":"2024-04-26T21:24:37.649628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:47.795571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132321+0000","last_change":"2024-04-26T21:30:53.835344+0000","last_active":"2024-04-26T21:30:55.132321+0000","last_peered":"2024-04-26T21:30:55.132321+0000","last_clean":"2024-04-26T21:30:55.132321+0000","last_became_active":"2024-04-26T21:30:53.834838+0000","last_became_peered":"2024-04-26T21:30:53.834838+0000","last_unstale":"2024-04-26T21:30:55.132321+0000","last_undegraded":"2024-04-26T21:30:55.132321+0000","last_fullsized":"2024-04-26T21:30:55.132321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:37.588955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.317390+0000","last_change":"2024-04-26T21:30:52.808173+0000","last_active":"2024-04-26T21:30:58.317390+0000","last_peered":"2024-04-26T21:30:58.317390+0000","last_clean":"2024-04-26T21:30:58.317390+0000","last_became_active":"2024-04-26T21:30:52.807688+0000","last_became_peered":"2024-04-26T21:30:52.807688+0000","last_unstale":"2024-04-26T21:30:58.317390+0000","last_undegraded":"2024-04-26T21:30:58.317390+0000","last_fullsized":"2024-04-26T21:30:58.317390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:58.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.343730+0000","last_change":"2024-04-26T21:30:52.810654+0000","last_active":"2024-04-26T21:31:00.343730+0000","last_peered":"2024-04-26T21:31:00.343730+0000","last_clean":"2024-04-26T21:31:00.343730+0000","last_became_active":"2024-04-26T21:30:52.808959+0000","last_became_peered":"2024-04-26T21:30:52.808959+0000","last_unstale":"2024-04-26T21:31:00.343730+0000","last_undegraded":"2024-04-26T21:31:00.343730+0000","last_fullsized":"2024-04-26T21:31:00.343730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:59.134325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132507+0000","last_change":"2024-04-26T21:30:53.816961+0000","last_active":"2024-04-26T21:30:55.132507+0000","last_peered":"2024-04-26T21:30:55.132507+0000","last_clean":"2024-04-26T21:30:55.132507+0000","last_became_active":"2024-04-26T21:30:53.816571+0000","last_became_peered":"2024-04-26T21:30:53.816571+0000","last_unstale":"2024-04-26T21:30:55.132507+0000","last_undegraded":"2024-04-26T21:30:55.132507+0000","last_fullsized":"2024-04-26T21:30:55.132507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:43.218823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.328208+0000","last_change":"2024-04-26T21:30:52.801027+0000","last_active":"2024-04-26T21:30:59.328208+0000","last_peered":"2024-04-26T21:30:59.328208+0000","last_clean":"2024-04-26T21:30:59.328208+0000","last_became_active":"2024-04-26T21:30:52.797107+0000","last_became_peered":"2024-04-26T21:30:52.797107+0000","last_unstale":"2024-04-26T21:30:59.328208+0000","last_undegraded":"2024-04-26T21:30:59.328208+0000","last_fullsized":"2024-04-26T21:30:59.328208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:23.739673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856823+0000","last_change":"2024-04-26T21:30:53.821946+0000","last_active":"2024-04-26T21:30:54.856823+0000","last_peered":"2024-04-26T21:30:54.856823+0000","last_clean":"2024-04-26T21:30:54.856823+0000","last_became_active":"2024-04-26T21:30:53.821433+0000","last_became_peered":"2024-04-26T21:30:53.821433+0000","last_unstale":"2024-04-26T21:30:54.856823+0000","last_undegraded":"2024-04-26T21:30:54.856823+0000","last_fullsized":"2024-04-26T21:30:54.856823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:30.426334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.297454+0000","last_change":"2024-04-26T21:30:52.806302+0000","last_active":"2024-04-26T21:30:57.297454+0000","last_peered":"2024-04-26T21:30:57.297454+0000","last_clean":"2024-04-26T21:30:57.297454+0000","last_became_active":"2024-04-26T21:30:52.803402+0000","last_became_peered":"2024-04-26T21:30:52.803402+0000","last_unstale":"2024-04-26T21:30:57.297454+0000","last_undegraded":"2024-04-26T21:30:57.297454+0000","last_fullsized":"2024-04-26T21:30:57.297454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:07.163372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133762+0000","last_change":"2024-04-26T21:30:53.833803+0000","last_active":"2024-04-26T21:30:55.133762+0000","last_peered":"2024-04-26T21:30:55.133762+0000","last_clean":"2024-04-26T21:30:55.133762+0000","last_became_active":"2024-04-26T21:30:53.833378+0000","last_became_peered":"2024-04-26T21:30:53.833378+0000","last_unstale":"2024-04-26T21:30:55.133762+0000","last_undegraded":"2024-04-26T21:30:55.133762+0000","last_fullsized":"2024-04-26T21:30:55.133762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:58.505849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.296991+0000","last_change":"2024-04-26T21:30:52.810031+0000","last_active":"2024-04-26T21:30:57.296991+0000","last_peered":"2024-04-26T21:30:57.296991+0000","last_clean":"2024-04-26T21:30:57.296991+0000","last_became_active":"2024-04-26T21:30:52.809743+0000","last_became_peered":"2024-04-26T21:30:52.809743+0000","last_unstale":"2024-04-26T21:30:57.296991+0000","last_undegraded":"2024-04-26T21:30:57.296991+0000","last_fullsized":"2024-04-26T21:30:57.296991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:05.723934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965099+0000","last_change":"2024-04-26T21:30:53.822563+0000","last_active":"2024-04-26T21:30:54.965099+0000","last_peered":"2024-04-26T21:30:54.965099+0000","last_clean":"2024-04-26T21:30:54.965099+0000","last_became_active":"2024-04-26T21:30:53.821848+0000","last_became_peered":"2024-04-26T21:30:53.821848+0000","last_unstale":"2024-04-26T21:30:54.965099+0000","last_undegraded":"2024-04-26T21:30:54.965099+0000","last_fullsized":"2024-04-26T21:30:54.965099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:39.861976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.296430+0000","last_change":"2024-04-26T21:30:52.792458+0000","last_active":"2024-04-26T21:30:57.296430+0000","last_peered":"2024-04-26T21:30:57.296430+0000","last_clean":"2024-04-26T21:30:57.296430+0000","last_became_active":"2024-04-26T21:30:52.792087+0000","last_became_peered":"2024-04-26T21:30:52.792087+0000","last_unstale":"2024-04-26T21:30:57.296430+0000","last_undegraded":"2024-04-26T21:30:57.296430+0000","last_fullsized":"2024-04-26T21:30:57.296430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:46.420509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795341+0000","last_change":"2024-04-26T21:30:53.826028+0000","last_active":"2024-04-26T21:30:54.795341+0000","last_peered":"2024-04-26T21:30:54.795341+0000","last_clean":"2024-04-26T21:30:54.795341+0000","last_became_active":"2024-04-26T21:30:53.822405+0000","last_became_peered":"2024-04-26T21:30:53.822405+0000","last_unstale":"2024-04-26T21:30:54.795341+0000","last_undegraded":"2024-04-26T21:30:54.795341+0000","last_fullsized":"2024-04-26T21:30:54.795341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:47.501074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.328237+0000","last_change":"2024-04-26T21:30:52.805708+0000","last_active":"2024-04-26T21:30:59.328237+0000","last_peered":"2024-04-26T21:30:59.328237+0000","last_clean":"2024-04-26T21:30:59.328237+0000","last_became_active":"2024-04-26T21:30:52.805258+0000","last_became_peered":"2024-04-26T21:30:52.805258+0000","last_unstale":"2024-04-26T21:30:59.328237+0000","last_undegraded":"2024-04-26T21:30:59.328237+0000","last_fullsized":"2024-04-26T21:30:59.328237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:58.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856322+0000","last_change":"2024-04-26T21:30:53.826016+0000","last_active":"2024-04-26T21:30:54.856322+0000","last_peered":"2024-04-26T21:30:54.856322+0000","last_clean":"2024-04-26T21:30:54.856322+0000","last_became_active":"2024-04-26T21:30:53.824050+0000","last_became_peered":"2024-04-26T21:30:53.824050+0000","last_unstale":"2024-04-26T21:30:54.856322+0000","last_undegraded":"2024-04-26T21:30:54.856322+0000","last_fullsized":"2024-04-26T21:30:54.856322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:35.816936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.301122+0000","last_change":"2024-04-26T21:30:52.811142+0000","last_active":"2024-04-26T21:30:57.301122+0000","last_peered":"2024-04-26T21:30:57.301122+0000","last_clean":"2024-04-26T21:30:57.301122+0000","last_became_active":"2024-04-26T21:30:52.810732+0000","last_became_peered":"2024-04-26T21:30:52.810732+0000","last_unstale":"2024-04-26T21:30:57.301122+0000","last_undegraded":"2024-04-26T21:30:57.301122+0000","last_fullsized":"2024-04-26T21:30:57.301122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:21.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975128+0000","last_change":"2024-04-26T21:30:53.832000+0000","last_active":"2024-04-26T21:30:54.975128+0000","last_peered":"2024-04-26T21:30:54.975128+0000","last_clean":"2024-04-26T21:30:54.975128+0000","last_became_active":"2024-04-26T21:30:53.831619+0000","last_became_peered":"2024-04-26T21:30:53.831619+0000","last_unstale":"2024-04-26T21:30:54.975128+0000","last_undegraded":"2024-04-26T21:30:54.975128+0000","last_fullsized":"2024-04-26T21:30:54.975128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:11.656424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.712535+0000","last_change":"2024-04-26T21:30:52.810311+0000","last_active":"2024-04-26T21:31:00.712535+0000","last_peered":"2024-04-26T21:31:00.712535+0000","last_clean":"2024-04-26T21:31:00.712535+0000","last_became_active":"2024-04-26T21:30:52.809976+0000","last_became_peered":"2024-04-26T21:30:52.809976+0000","last_unstale":"2024-04-26T21:31:00.712535+0000","last_undegraded":"2024-04-26T21:31:00.712535+0000","last_fullsized":"2024-04-26T21:31:00.712535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:49.146730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975123+0000","last_change":"2024-04-26T21:30:53.830315+0000","last_active":"2024-04-26T21:30:54.975123+0000","last_peered":"2024-04-26T21:30:54.975123+0000","last_clean":"2024-04-26T21:30:54.975123+0000","last_became_active":"2024-04-26T21:30:53.829439+0000","last_became_peered":"2024-04-26T21:30:53.829439+0000","last_unstale":"2024-04-26T21:30:54.975123+0000","last_undegraded":"2024-04-26T21:30:54.975123+0000","last_fullsized":"2024-04-26T21:30:54.975123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:49.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133896+0000","last_change":"2024-04-26T21:30:52.802392+0000","last_active":"2024-04-26T21:30:55.133896+0000","last_peered":"2024-04-26T21:30:55.133896+0000","last_clean":"2024-04-26T21:30:55.133896+0000","last_became_active":"2024-04-26T21:30:52.799978+0000","last_became_peered":"2024-04-26T21:30:52.799978+0000","last_unstale":"2024-04-26T21:30:55.133896+0000","last_undegraded":"2024-04-26T21:30:55.133896+0000","last_fullsized":"2024-04-26T21:30:55.133896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:45.993473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975106+0000","last_change":"2024-04-26T21:30:53.818118+0000","last_active":"2024-04-26T21:30:54.975106+0000","last_peered":"2024-04-26T21:30:54.975106+0000","last_clean":"2024-04-26T21:30:54.975106+0000","last_became_active":"2024-04-26T21:30:53.817712+0000","last_became_peered":"2024-04-26T21:30:53.817712+0000","last_unstale":"2024-04-26T21:30:54.975106+0000","last_undegraded":"2024-04-26T21:30:54.975106+0000","last_fullsized":"2024-04-26T21:30:54.975106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.819569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133542+0000","last_change":"2024-04-26T21:30:52.807308+0000","last_active":"2024-04-26T21:30:55.133542+0000","last_peered":"2024-04-26T21:30:55.133542+0000","last_clean":"2024-04-26T21:30:55.133542+0000","last_became_active":"2024-04-26T21:30:52.806935+0000","last_became_peered":"2024-04-26T21:30:52.806935+0000","last_unstale":"2024-04-26T21:30:55.133542+0000","last_undegraded":"2024-04-26T21:30:55.133542+0000","last_fullsized":"2024-04-26T21:30:55.133542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:42.549564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975460+0000","last_change":"2024-04-26T21:30:53.812940+0000","last_active":"2024-04-26T21:30:54.975460+0000","last_peered":"2024-04-26T21:30:54.975460+0000","last_clean":"2024-04-26T21:30:54.975460+0000","last_became_active":"2024-04-26T21:30:53.812492+0000","last_became_peered":"2024-04-26T21:30:53.812492+0000","last_unstale":"2024-04-26T21:30:54.975460+0000","last_undegraded":"2024-04-26T21:30:54.975460+0000","last_fullsized":"2024-04-26T21:30:54.975460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:07.531226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795328+0000","last_change":"2024-04-26T21:30:53.822943+0000","last_active":"2024-04-26T21:30:54.795328+0000","last_peered":"2024-04-26T21:30:54.795328+0000","last_clean":"2024-04-26T21:30:54.795328+0000","last_became_active":"2024-04-26T21:30:53.822443+0000","last_became_peered":"2024-04-26T21:30:53.822443+0000","last_unstale":"2024-04-26T21:30:54.795328+0000","last_undegraded":"2024-04-26T21:30:54.795328+0000","last_fullsized":"2024-04-26T21:30:54.795328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:57.827470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133049+0000","last_change":"2024-04-26T21:30:53.832417+0000","last_active":"2024-04-26T21:30:55.133049+0000","last_peered":"2024-04-26T21:30:55.133049+0000","last_clean":"2024-04-26T21:30:55.133049+0000","last_became_active":"2024-04-26T21:30:53.832001+0000","last_became_peered":"2024-04-26T21:30:53.832001+0000","last_unstale":"2024-04-26T21:30:55.133049+0000","last_undegraded":"2024-04-26T21:30:55.133049+0000","last_fullsized":"2024-04-26T21:30:55.133049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:02.471766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134139+0000","last_change":"2024-04-26T21:30:53.817472+0000","last_active":"2024-04-26T21:30:55.134139+0000","last_peered":"2024-04-26T21:30:55.134139+0000","last_clean":"2024-04-26T21:30:55.134139+0000","last_became_active":"2024-04-26T21:30:53.817053+0000","last_became_peered":"2024-04-26T21:30:53.817053+0000","last_unstale":"2024-04-26T21:30:55.134139+0000","last_undegraded":"2024-04-26T21:30:55.134139+0000","last_fullsized":"2024-04-26T21:30:55.134139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:19.219132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133559+0000","last_change":"2024-04-26T21:30:53.826748+0000","last_active":"2024-04-26T21:30:55.133559+0000","last_peered":"2024-04-26T21:30:55.133559+0000","last_clean":"2024-04-26T21:30:55.133559+0000","last_became_active":"2024-04-26T21:30:53.826040+0000","last_became_peered":"2024-04-26T21:30:53.826040+0000","last_unstale":"2024-04-26T21:30:55.133559+0000","last_undegraded":"2024-04-26T21:30:55.133559+0000","last_fullsized":"2024-04-26T21:30:55.133559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:18.222414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132603+0000","last_change":"2024-04-26T21:30:53.825506+0000","last_active":"2024-04-26T21:30:55.132603+0000","last_peered":"2024-04-26T21:30:55.132603+0000","last_clean":"2024-04-26T21:30:55.132603+0000","last_became_active":"2024-04-26T21:30:53.824162+0000","last_became_peered":"2024-04-26T21:30:53.824162+0000","last_unstale":"2024-04-26T21:30:55.132603+0000","last_undegraded":"2024-04-26T21:30:55.132603+0000","last_fullsized":"2024-04-26T21:30:55.132603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:58.088580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795889+0000","last_change":"2024-04-26T21:30:53.825396+0000","last_active":"2024-04-26T21:30:54.795889+0000","last_peered":"2024-04-26T21:30:54.795889+0000","last_clean":"2024-04-26T21:30:54.795889+0000","last_became_active":"2024-04-26T21:30:53.825042+0000","last_became_peered":"2024-04-26T21:30:53.825042+0000","last_unstale":"2024-04-26T21:30:54.795889+0000","last_undegraded":"2024-04-26T21:30:54.795889+0000","last_fullsized":"2024-04-26T21:30:54.795889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:20.064265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134033+0000","last_change":"2024-04-26T21:30:53.827819+0000","last_active":"2024-04-26T21:30:55.134033+0000","last_peered":"2024-04-26T21:30:55.134033+0000","last_clean":"2024-04-26T21:30:55.134033+0000","last_became_active":"2024-04-26T21:30:53.827027+0000","last_became_peered":"2024-04-26T21:30:53.827027+0000","last_unstale":"2024-04-26T21:30:55.134033+0000","last_undegraded":"2024-04-26T21:30:55.134033+0000","last_fullsized":"2024-04-26T21:30:55.134033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:49.586652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856701+0000","last_change":"2024-04-26T21:30:53.824693+0000","last_active":"2024-04-26T21:30:54.856701+0000","last_peered":"2024-04-26T21:30:54.856701+0000","last_clean":"2024-04-26T21:30:54.856701+0000","last_became_active":"2024-04-26T21:30:53.824278+0000","last_became_peered":"2024-04-26T21:30:53.824278+0000","last_unstale":"2024-04-26T21:30:54.856701+0000","last_undegraded":"2024-04-26T21:30:54.856701+0000","last_fullsized":"2024-04-26T21:30:54.856701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:42.799139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133392+0000","last_change":"2024-04-26T21:30:53.823886+0000","last_active":"2024-04-26T21:30:55.133392+0000","last_peered":"2024-04-26T21:30:55.133392+0000","last_clean":"2024-04-26T21:30:55.133392+0000","last_became_active":"2024-04-26T21:30:53.823443+0000","last_became_peered":"2024-04-26T21:30:53.823443+0000","last_unstale":"2024-04-26T21:30:55.133392+0000","last_undegraded":"2024-04-26T21:30:55.133392+0000","last_fullsized":"2024-04-26T21:30:55.133392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.022073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856151+0000","last_change":"2024-04-26T21:30:53.825329+0000","last_active":"2024-04-26T21:30:54.856151+0000","last_peered":"2024-04-26T21:30:54.856151+0000","last_clean":"2024-04-26T21:30:54.856151+0000","last_became_active":"2024-04-26T21:30:53.822833+0000","last_became_peered":"2024-04-26T21:30:53.822833+0000","last_unstale":"2024-04-26T21:30:54.856151+0000","last_undegraded":"2024-04-26T21:30:54.856151+0000","last_fullsized":"2024-04-26T21:30:54.856151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.840177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.135530+0000","last_change":"2024-04-26T21:30:53.837657+0000","last_active":"2024-04-26T21:30:55.135530+0000","last_peered":"2024-04-26T21:30:55.135530+0000","last_clean":"2024-04-26T21:30:55.135530+0000","last_became_active":"2024-04-26T21:30:53.837354+0000","last_became_peered":"2024-04-26T21:30:53.837354+0000","last_unstale":"2024-04-26T21:30:55.135530+0000","last_undegraded":"2024-04-26T21:30:55.135530+0000","last_fullsized":"2024-04-26T21:30:55.135530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:34.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132680+0000","last_change":"2024-04-26T21:30:53.811372+0000","last_active":"2024-04-26T21:30:55.132680+0000","last_peered":"2024-04-26T21:30:55.132680+0000","last_clean":"2024-04-26T21:30:55.132680+0000","last_became_active":"2024-04-26T21:30:53.810124+0000","last_became_peered":"2024-04-26T21:30:53.810124+0000","last_unstale":"2024-04-26T21:30:55.132680+0000","last_undegraded":"2024-04-26T21:30:55.132680+0000","last_fullsized":"2024-04-26T21:30:55.132680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:14.134041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795371+0000","last_change":"2024-04-26T21:30:53.836391+0000","last_active":"2024-04-26T21:30:54.795371+0000","last_peered":"2024-04-26T21:30:54.795371+0000","last_clean":"2024-04-26T21:30:54.795371+0000","last_became_active":"2024-04-26T21:30:53.835987+0000","last_became_peered":"2024-04-26T21:30:53.835987+0000","last_unstale":"2024-04-26T21:30:54.795371+0000","last_undegraded":"2024-04-26T21:30:54.795371+0000","last_fullsized":"2024-04-26T21:30:54.795371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:56.449077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26302,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":184,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":58423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547224,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1045929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710751,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":452764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972391,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":453356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266733,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1044688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":451856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":475505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":452050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":452522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1552384,"data_stored":1042406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:31:03.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:31:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:03 smithi143 ceph-mon[31461]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:03 smithi143 ceph-mon[31461]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:03 smithi143 ceph-mon[31461]: pgmap v369: 97 pgs: 97 active+clean; 602 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-26T21:31:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:03 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2042943131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:03 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3259459844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:03 smithi097 ceph-mon[28127]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:03 smithi097 ceph-mon[28127]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:03 smithi097 ceph-mon[28127]: pgmap v369: 97 pgs: 97 active+clean; 602 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-26T21:31:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:03 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2042943131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:03 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3259459844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.075+0000 7f3397c00640 1 -- 172.21.15.97:0/1418484066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390106740 msgr2=0x7f339010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.075+0000 7f3397c00640 1 --2- 172.21.15.97:0/1418484066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390106740 0x7f339010cfd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3388009ca0 tx=0x7f338802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 -- 172.21.15.97:0/1418484066 shutdown_connections 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 --2- 172.21.15.97:0/1418484066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390106740 0x7f339010cfd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 --2- 172.21.15.97:0/1418484066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390105d80 0x7f3390106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 --2- 172.21.15.97:0/1418484066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3390104b90 0x7f3390104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 -- 172.21.15.97:0/1418484066 >> 172.21.15.97:0/1418484066 conn(0x7f3390100120 msgr2=0x7f3390102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 -- 172.21.15.97:0/1418484066 shutdown_connections 2024-04-26T21:31:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 -- 172.21.15.97:0/1418484066 wait complete. 2024-04-26T21:31:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.076+0000 7f3397c00640 1 Processor -- start 2024-04-26T21:31:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 -- start start 2024-04-26T21:31:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390104b90 0x7f339019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3390106740 0x7f33901a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33901105e0 con 0x7f3390105d80 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3390110460 con 0x7f3390106740 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3397c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3390110760 con 0x7f3390104b90 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f338fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f338fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36144/0 (socket says 172.21.15.97:36144) 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f338fe00640 1 -- 172.21.15.97:0/2496229744 learned_addr learned my addr 172.21.15.97:0/2496229744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f3395200640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390104b90 0x7f339019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.077+0000 7f338fe00640 1 -- 172.21.15.97:0/2496229744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390104b90 msgr2=0x7f339019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338fe00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390104b90 0x7f339019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338fe00640 1 -- 172.21.15.97:0/2496229744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3390106740 msgr2=0x7f33901a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338fe00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3390106740 0x7f33901a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338fe00640 1 -- 172.21.15.97:0/2496229744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33901a28d0 con 0x7f3390105d80 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338fe00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f337c00d770 tx=0x7f337c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c017070 con 0x7f3390105d80 2024-04-26T21:31:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337c0054b0 con 0x7f3390105d80 2024-04-26T21:31:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33901a2bc0 con 0x7f3390105d80 2024-04-26T21:31:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c004c60 con 0x7f3390105d80 2024-04-26T21:31:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.078+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901a30c0 con 0x7f3390105d80 2024-04-26T21:31:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.079+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339010bf30 con 0x7f3390105d80 2024-04-26T21:31:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.081+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f337c004e00 con 0x7f3390105d80 2024-04-26T21:31:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.082+0000 7f338d600640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 0x7f33780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.083+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f337c0cfff0 con 0x7f3390105d80 2024-04-26T21:31:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.083+0000 7f3395200640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 0x7f33780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.083+0000 7f3395200640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 0x7f33780a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3384005fd0 tx=0x7f3384005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.086+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f337c09c920 con 0x7f3390105d80 2024-04-26T21:31:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:03 smithi111 ceph-mon[31001]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:03 smithi111 ceph-mon[31001]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:31:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:03 smithi111 ceph-mon[31001]: pgmap v369: 97 pgs: 97 active+clean; 602 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-26T21:31:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:03 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2042943131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:03 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3259459844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:04.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.282+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3390062d70 con 0x7f33780a3c40 2024-04-26T21:31:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.293+0000 7f338d600640 1 -- 172.21.15.97:0/2496229744 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258853 (secure 0 0 0) 0x7f3390062d70 con 0x7f33780a3c40 2024-04-26T21:31:04.298 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:04.299 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-26T21:31:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 msgr2=0x7f33780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 0x7f33780a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3384005fd0 tx=0x7f3384005e40 comp rx=0 tx=0).stop 2024-04-26T21:31:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 msgr2=0x7f339019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f337c00d770 tx=0x7f337c00dc30 comp rx=0 tx=0).stop 2024-04-26T21:31:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 shutdown_connections 2024-04-26T21:31:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3390106740 0x7f33901a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f33780a3c40 0x7f33780a6100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390105d80 0x7f339019b1d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 --2- 172.21.15.97:0/2496229744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390104b90 0x7f339019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.299+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 >> 172.21.15.97:0/2496229744 conn(0x7f3390100120 msgr2=0x7f3390109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.300+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 shutdown_connections 2024-04-26T21:31:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:04.300+0000 7f3397c00640 1 -- 172.21.15.97:0/2496229744 wait complete. 2024-04-26T21:31:04.320 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-26T21:31:02.654565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616787,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":145,"num_read_kb":133,"num_write":366,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391392,"kb_used_data":13104,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550432,"statfs":{"total":1151940427776,"available":1151539642368,"internally_reserved":0,"allocated":13418496,"data_stored":7230316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26419,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":185,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.864643"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796262+0000","last_change":"2024-04-26T21:30:53.833872+0000","last_active":"2024-04-26T21:30:54.796262+0000","last_peered":"2024-04-26T21:30:54.796262+0000","last_clean":"2024-04-26T21:30:54.796262+0000","last_became_active":"2024-04-26T21:30:53.832109+0000","last_became_peered":"2024-04-26T21:30:53.832109+0000","last_unstale":"2024-04-26T21:30:54.796262+0000","last_undegraded":"2024-04-26T21:30:54.796262+0000","last_fullsized":"2024-04-26T21:30:54.796262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:50.076505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133733+0000","last_change":"2024-04-26T21:30:53.827034+0000","last_active":"2024-04-26T21:30:55.133733+0000","last_peered":"2024-04-26T21:30:55.133733+0000","last_clean":"2024-04-26T21:30:55.133733+0000","last_became_active":"2024-04-26T21:30:53.826682+0000","last_became_peered":"2024-04-26T21:30:53.826682+0000","last_unstale":"2024-04-26T21:30:55.133733+0000","last_undegraded":"2024-04-26T21:30:55.133733+0000","last_fullsized":"2024-04-26T21:30:55.133733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:39.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133748+0000","last_change":"2024-04-26T21:30:53.833196+0000","last_active":"2024-04-26T21:30:55.133748+0000","last_peered":"2024-04-26T21:30:55.133748+0000","last_clean":"2024-04-26T21:30:55.133748+0000","last_became_active":"2024-04-26T21:30:53.831214+0000","last_became_peered":"2024-04-26T21:30:53.831214+0000","last_unstale":"2024-04-26T21:30:55.133748+0000","last_undegraded":"2024-04-26T21:30:55.133748+0000","last_fullsized":"2024-04-26T21:30:55.133748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:17.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133925+0000","last_change":"2024-04-26T21:30:53.829249+0000","last_active":"2024-04-26T21:30:55.133925+0000","last_peered":"2024-04-26T21:30:55.133925+0000","last_clean":"2024-04-26T21:30:55.133925+0000","last_became_active":"2024-04-26T21:30:53.828170+0000","last_became_peered":"2024-04-26T21:30:53.828170+0000","last_unstale":"2024-04-26T21:30:55.133925+0000","last_undegraded":"2024-04-26T21:30:55.133925+0000","last_fullsized":"2024-04-26T21:30:55.133925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:30.729567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965809+0000","last_change":"2024-04-26T21:30:53.824137+0000","last_active":"2024-04-26T21:30:54.965809+0000","last_peered":"2024-04-26T21:30:54.965809+0000","last_clean":"2024-04-26T21:30:54.965809+0000","last_became_active":"2024-04-26T21:30:53.822807+0000","last_became_peered":"2024-04-26T21:30:53.822807+0000","last_unstale":"2024-04-26T21:30:54.965809+0000","last_undegraded":"2024-04-26T21:30:54.965809+0000","last_fullsized":"2024-04-26T21:30:54.965809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:03.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.794932+0000","last_change":"2024-04-26T21:30:53.835932+0000","last_active":"2024-04-26T21:30:54.794932+0000","last_peered":"2024-04-26T21:30:54.794932+0000","last_clean":"2024-04-26T21:30:54.794932+0000","last_became_active":"2024-04-26T21:30:53.835500+0000","last_became_peered":"2024-04-26T21:30:53.835500+0000","last_unstale":"2024-04-26T21:30:54.794932+0000","last_undegraded":"2024-04-26T21:30:54.794932+0000","last_fullsized":"2024-04-26T21:30:54.794932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:34.312999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975197+0000","last_change":"2024-04-26T21:30:53.815175+0000","last_active":"2024-04-26T21:30:54.975197+0000","last_peered":"2024-04-26T21:30:54.975197+0000","last_clean":"2024-04-26T21:30:54.975197+0000","last_became_active":"2024-04-26T21:30:53.814774+0000","last_became_peered":"2024-04-26T21:30:53.814774+0000","last_unstale":"2024-04-26T21:30:54.975197+0000","last_undegraded":"2024-04-26T21:30:54.975197+0000","last_fullsized":"2024-04-26T21:30:54.975197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:21.815872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856133+0000","last_change":"2024-04-26T21:30:53.824718+0000","last_active":"2024-04-26T21:30:54.856133+0000","last_peered":"2024-04-26T21:30:54.856133+0000","last_clean":"2024-04-26T21:30:54.856133+0000","last_became_active":"2024-04-26T21:30:53.824372+0000","last_became_peered":"2024-04-26T21:30:53.824372+0000","last_unstale":"2024-04-26T21:30:54.856133+0000","last_undegraded":"2024-04-26T21:30:54.856133+0000","last_fullsized":"2024-04-26T21:30:54.856133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:33.450906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136604+0000","last_change":"2024-04-26T21:30:53.830237+0000","last_active":"2024-04-26T21:30:55.136604+0000","last_peered":"2024-04-26T21:30:55.136604+0000","last_clean":"2024-04-26T21:30:55.136604+0000","last_became_active":"2024-04-26T21:30:53.829837+0000","last_became_peered":"2024-04-26T21:30:53.829837+0000","last_unstale":"2024-04-26T21:30:55.136604+0000","last_undegraded":"2024-04-26T21:30:55.136604+0000","last_fullsized":"2024-04-26T21:30:55.136604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:33:15.161046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795317+0000","last_change":"2024-04-26T21:30:53.838240+0000","last_active":"2024-04-26T21:30:54.795317+0000","last_peered":"2024-04-26T21:30:54.795317+0000","last_clean":"2024-04-26T21:30:54.795317+0000","last_became_active":"2024-04-26T21:30:53.837741+0000","last_became_peered":"2024-04-26T21:30:53.837741+0000","last_unstale":"2024-04-26T21:30:54.795317+0000","last_undegraded":"2024-04-26T21:30:54.795317+0000","last_fullsized":"2024-04-26T21:30:54.795317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:33.879043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975123+0000","last_change":"2024-04-26T21:30:53.830564+0000","last_active":"2024-04-26T21:30:54.975123+0000","last_peered":"2024-04-26T21:30:54.975123+0000","last_clean":"2024-04-26T21:30:54.975123+0000","last_became_active":"2024-04-26T21:30:53.829967+0000","last_became_peered":"2024-04-26T21:30:53.829967+0000","last_unstale":"2024-04-26T21:30:54.975123+0000","last_undegraded":"2024-04-26T21:30:54.975123+0000","last_fullsized":"2024-04-26T21:30:54.975123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:06.931219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133668+0000","last_change":"2024-04-26T21:30:53.829434+0000","last_active":"2024-04-26T21:30:55.133668+0000","last_peered":"2024-04-26T21:30:55.133668+0000","last_clean":"2024-04-26T21:30:55.133668+0000","last_became_active":"2024-04-26T21:30:53.828657+0000","last_became_peered":"2024-04-26T21:30:53.828657+0000","last_unstale":"2024-04-26T21:30:55.133668+0000","last_undegraded":"2024-04-26T21:30:55.133668+0000","last_fullsized":"2024-04-26T21:30:55.133668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:12.215856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133005+0000","last_change":"2024-04-26T21:30:53.824833+0000","last_active":"2024-04-26T21:30:55.133005+0000","last_peered":"2024-04-26T21:30:55.133005+0000","last_clean":"2024-04-26T21:30:55.133005+0000","last_became_active":"2024-04-26T21:30:53.824489+0000","last_became_peered":"2024-04-26T21:30:53.824489+0000","last_unstale":"2024-04-26T21:30:55.133005+0000","last_undegraded":"2024-04-26T21:30:55.133005+0000","last_fullsized":"2024-04-26T21:30:55.133005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:48.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132977+0000","last_change":"2024-04-26T21:30:53.824573+0000","last_active":"2024-04-26T21:30:55.132977+0000","last_peered":"2024-04-26T21:30:55.132977+0000","last_clean":"2024-04-26T21:30:55.132977+0000","last_became_active":"2024-04-26T21:30:53.824242+0000","last_became_peered":"2024-04-26T21:30:53.824242+0000","last_unstale":"2024-04-26T21:30:55.132977+0000","last_undegraded":"2024-04-26T21:30:55.132977+0000","last_fullsized":"2024-04-26T21:30:55.132977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:12.271919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133194+0000","last_change":"2024-04-26T21:30:53.827619+0000","last_active":"2024-04-26T21:30:55.133194+0000","last_peered":"2024-04-26T21:30:55.133194+0000","last_clean":"2024-04-26T21:30:55.133194+0000","last_became_active":"2024-04-26T21:30:53.827301+0000","last_became_peered":"2024-04-26T21:30:53.827301+0000","last_unstale":"2024-04-26T21:30:55.133194+0000","last_undegraded":"2024-04-26T21:30:55.133194+0000","last_fullsized":"2024-04-26T21:30:55.133194+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:25.152302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134868+0000","last_change":"2024-04-26T21:30:53.825785+0000","last_active":"2024-04-26T21:30:55.134868+0000","last_peered":"2024-04-26T21:30:55.134868+0000","last_clean":"2024-04-26T21:30:55.134868+0000","last_became_active":"2024-04-26T21:30:53.821673+0000","last_became_peered":"2024-04-26T21:30:53.821673+0000","last_unstale":"2024-04-26T21:30:55.134868+0000","last_undegraded":"2024-04-26T21:30:55.134868+0000","last_fullsized":"2024-04-26T21:30:55.134868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:39.138995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796284+0000","last_change":"2024-04-26T21:30:53.834768+0000","last_active":"2024-04-26T21:30:54.796284+0000","last_peered":"2024-04-26T21:30:54.796284+0000","last_clean":"2024-04-26T21:30:54.796284+0000","last_became_active":"2024-04-26T21:30:53.831624+0000","last_became_peered":"2024-04-26T21:30:53.831624+0000","last_unstale":"2024-04-26T21:30:54.796284+0000","last_undegraded":"2024-04-26T21:30:54.796284+0000","last_fullsized":"2024-04-26T21:30:54.796284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:49.228987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.797327+0000","last_change":"2024-04-26T21:30:53.828348+0000","last_active":"2024-04-26T21:30:54.797327+0000","last_peered":"2024-04-26T21:30:54.797327+0000","last_clean":"2024-04-26T21:30:54.797327+0000","last_became_active":"2024-04-26T21:30:53.827926+0000","last_became_peered":"2024-04-26T21:30:53.827926+0000","last_unstale":"2024-04-26T21:30:54.797327+0000","last_undegraded":"2024-04-26T21:30:54.797327+0000","last_fullsized":"2024-04-26T21:30:54.797327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133966+0000","last_change":"2024-04-26T21:30:53.824205+0000","last_active":"2024-04-26T21:30:55.133966+0000","last_peered":"2024-04-26T21:30:55.133966+0000","last_clean":"2024-04-26T21:30:55.133966+0000","last_became_active":"2024-04-26T21:30:53.823787+0000","last_became_peered":"2024-04-26T21:30:53.823787+0000","last_unstale":"2024-04-26T21:30:55.133966+0000","last_undegraded":"2024-04-26T21:30:55.133966+0000","last_fullsized":"2024-04-26T21:30:55.133966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:23.325924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'11","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.325361+0000","last_change":"2024-04-26T21:30:52.803835+0000","last_active":"2024-04-26T21:30:59.325361+0000","last_peered":"2024-04-26T21:30:59.325361+0000","last_clean":"2024-04-26T21:30:59.325361+0000","last_became_active":"2024-04-26T21:30:52.803541+0000","last_became_peered":"2024-04-26T21:30:52.803541+0000","last_unstale":"2024-04-26T21:30:59.325361+0000","last_undegraded":"2024-04-26T21:30:59.325361+0000","last_fullsized":"2024-04-26T21:30:59.325361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:16.314776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856814+0000","last_change":"2024-04-26T21:30:53.823811+0000","last_active":"2024-04-26T21:30:54.856814+0000","last_peered":"2024-04-26T21:30:54.856814+0000","last_clean":"2024-04-26T21:30:54.856814+0000","last_became_active":"2024-04-26T21:30:53.823284+0000","last_became_peered":"2024-04-26T21:30:53.823284+0000","last_unstale":"2024-04-26T21:30:54.856814+0000","last_undegraded":"2024-04-26T21:30:54.856814+0000","last_fullsized":"2024-04-26T21:30:54.856814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:05.627445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856825+0000","last_change":"2024-04-26T21:30:52.810994+0000","last_active":"2024-04-26T21:30:54.856825+0000","last_peered":"2024-04-26T21:30:54.856825+0000","last_clean":"2024-04-26T21:30:54.856825+0000","last_became_active":"2024-04-26T21:30:52.809509+0000","last_became_peered":"2024-04-26T21:30:52.809509+0000","last_unstale":"2024-04-26T21:30:54.856825+0000","last_undegraded":"2024-04-26T21:30:54.856825+0000","last_fullsized":"2024-04-26T21:30:54.856825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:24.366289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133952+0000","last_change":"2024-04-26T21:30:53.830471+0000","last_active":"2024-04-26T21:30:55.133952+0000","last_peered":"2024-04-26T21:30:55.133952+0000","last_clean":"2024-04-26T21:30:55.133952+0000","last_became_active":"2024-04-26T21:30:53.829841+0000","last_became_peered":"2024-04-26T21:30:53.829841+0000","last_unstale":"2024-04-26T21:30:55.133952+0000","last_undegraded":"2024-04-26T21:30:55.133952+0000","last_fullsized":"2024-04-26T21:30:55.133952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:25.988137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.705675+0000","last_change":"2024-04-26T21:30:52.812564+0000","last_active":"2024-04-26T21:31:00.705675+0000","last_peered":"2024-04-26T21:31:00.705675+0000","last_clean":"2024-04-26T21:31:00.705675+0000","last_became_active":"2024-04-26T21:30:52.811090+0000","last_became_peered":"2024-04-26T21:30:52.811090+0000","last_unstale":"2024-04-26T21:31:00.705675+0000","last_undegraded":"2024-04-26T21:31:00.705675+0000","last_fullsized":"2024-04-26T21:31:00.705675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:51.522689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.135859+0000","last_change":"2024-04-26T21:30:53.828398+0000","last_active":"2024-04-26T21:30:55.135859+0000","last_peered":"2024-04-26T21:30:55.135859+0000","last_clean":"2024-04-26T21:30:55.135859+0000","last_became_active":"2024-04-26T21:30:53.827920+0000","last_became_peered":"2024-04-26T21:30:53.827920+0000","last_unstale":"2024-04-26T21:30:55.135859+0000","last_undegraded":"2024-04-26T21:30:55.135859+0000","last_fullsized":"2024-04-26T21:30:55.135859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:06.463121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.341294+0000","last_change":"2024-04-26T21:30:52.798297+0000","last_active":"2024-04-26T21:31:00.341294+0000","last_peered":"2024-04-26T21:31:00.341294+0000","last_clean":"2024-04-26T21:31:00.341294+0000","last_became_active":"2024-04-26T21:30:52.797998+0000","last_became_peered":"2024-04-26T21:30:52.797998+0000","last_unstale":"2024-04-26T21:31:00.341294+0000","last_undegraded":"2024-04-26T21:31:00.341294+0000","last_fullsized":"2024-04-26T21:31:00.341294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:40.790353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134028+0000","last_change":"2024-04-26T21:30:53.830846+0000","last_active":"2024-04-26T21:30:55.134028+0000","last_peered":"2024-04-26T21:30:55.134028+0000","last_clean":"2024-04-26T21:30:55.134028+0000","last_became_active":"2024-04-26T21:30:53.830294+0000","last_became_peered":"2024-04-26T21:30:53.830294+0000","last_unstale":"2024-04-26T21:30:55.134028+0000","last_undegraded":"2024-04-26T21:30:55.134028+0000","last_fullsized":"2024-04-26T21:30:55.134028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:50.929819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.301190+0000","last_change":"2024-04-26T21:30:52.807155+0000","last_active":"2024-04-26T21:30:57.301190+0000","last_peered":"2024-04-26T21:30:57.301190+0000","last_clean":"2024-04-26T21:30:57.301190+0000","last_became_active":"2024-04-26T21:30:52.805230+0000","last_became_peered":"2024-04-26T21:30:52.805230+0000","last_unstale":"2024-04-26T21:30:57.301190+0000","last_undegraded":"2024-04-26T21:30:57.301190+0000","last_fullsized":"2024-04-26T21:30:57.301190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:44.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965493+0000","last_change":"2024-04-26T21:30:53.810334+0000","last_active":"2024-04-26T21:30:54.965493+0000","last_peered":"2024-04-26T21:30:54.965493+0000","last_clean":"2024-04-26T21:30:54.965493+0000","last_became_active":"2024-04-26T21:30:53.809867+0000","last_became_peered":"2024-04-26T21:30:53.809867+0000","last_unstale":"2024-04-26T21:30:54.965493+0000","last_undegraded":"2024-04-26T21:30:54.965493+0000","last_fullsized":"2024-04-26T21:30:54.965493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:46.377262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.327103+0000","last_change":"2024-04-26T21:30:52.804775+0000","last_active":"2024-04-26T21:30:59.327103+0000","last_peered":"2024-04-26T21:30:59.327103+0000","last_clean":"2024-04-26T21:30:59.327103+0000","last_became_active":"2024-04-26T21:30:52.804319+0000","last_became_peered":"2024-04-26T21:30:52.804319+0000","last_unstale":"2024-04-26T21:30:59.327103+0000","last_undegraded":"2024-04-26T21:30:59.327103+0000","last_fullsized":"2024-04-26T21:30:59.327103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:12.761080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133866+0000","last_change":"2024-04-26T21:30:53.823021+0000","last_active":"2024-04-26T21:30:55.133866+0000","last_peered":"2024-04-26T21:30:55.133866+0000","last_clean":"2024-04-26T21:30:55.133866+0000","last_became_active":"2024-04-26T21:30:53.822633+0000","last_became_peered":"2024-04-26T21:30:53.822633+0000","last_unstale":"2024-04-26T21:30:55.133866+0000","last_undegraded":"2024-04-26T21:30:55.133866+0000","last_fullsized":"2024-04-26T21:30:55.133866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:32.178472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.314848+0000","last_change":"2024-04-26T21:30:52.796729+0000","last_active":"2024-04-26T21:30:58.314848+0000","last_peered":"2024-04-26T21:30:58.314848+0000","last_clean":"2024-04-26T21:30:58.314848+0000","last_became_active":"2024-04-26T21:30:52.796341+0000","last_became_peered":"2024-04-26T21:30:52.796341+0000","last_unstale":"2024-04-26T21:30:58.314848+0000","last_undegraded":"2024-04-26T21:30:58.314848+0000","last_fullsized":"2024-04-26T21:30:58.314848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:30.434541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.855935+0000","last_change":"2024-04-26T21:30:53.816136+0000","last_active":"2024-04-26T21:30:54.855935+0000","last_peered":"2024-04-26T21:30:54.855935+0000","last_clean":"2024-04-26T21:30:54.855935+0000","last_became_active":"2024-04-26T21:30:53.815758+0000","last_became_peered":"2024-04-26T21:30:53.815758+0000","last_unstale":"2024-04-26T21:30:54.855935+0000","last_undegraded":"2024-04-26T21:30:54.855935+0000","last_fullsized":"2024-04-26T21:30:54.855935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:41.606483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133180+0000","last_change":"2024-04-26T21:30:52.804102+0000","last_active":"2024-04-26T21:30:55.133180+0000","last_peered":"2024-04-26T21:30:55.133180+0000","last_clean":"2024-04-26T21:30:55.133180+0000","last_became_active":"2024-04-26T21:30:52.803808+0000","last_became_peered":"2024-04-26T21:30:52.803808+0000","last_unstale":"2024-04-26T21:30:55.133180+0000","last_undegraded":"2024-04-26T21:30:55.133180+0000","last_fullsized":"2024-04-26T21:30:55.133180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:16.357651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.796196+0000","last_change":"2024-04-26T21:30:53.822475+0000","last_active":"2024-04-26T21:30:54.796196+0000","last_peered":"2024-04-26T21:30:54.796196+0000","last_clean":"2024-04-26T21:30:54.796196+0000","last_became_active":"2024-04-26T21:30:53.822077+0000","last_became_peered":"2024-04-26T21:30:53.822077+0000","last_unstale":"2024-04-26T21:30:54.796196+0000","last_undegraded":"2024-04-26T21:30:54.796196+0000","last_fullsized":"2024-04-26T21:30:54.796196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:09.289557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.708579+0000","last_change":"2024-04-26T21:30:52.802080+0000","last_active":"2024-04-26T21:31:00.708579+0000","last_peered":"2024-04-26T21:31:00.708579+0000","last_clean":"2024-04-26T21:31:00.708579+0000","last_became_active":"2024-04-26T21:30:52.801645+0000","last_became_peered":"2024-04-26T21:30:52.801645+0000","last_unstale":"2024-04-26T21:31:00.708579+0000","last_undegraded":"2024-04-26T21:31:00.708579+0000","last_fullsized":"2024-04-26T21:31:00.708579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:00.213924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136085+0000","last_change":"2024-04-26T21:30:53.832936+0000","last_active":"2024-04-26T21:30:55.136085+0000","last_peered":"2024-04-26T21:30:55.136085+0000","last_clean":"2024-04-26T21:30:55.136085+0000","last_became_active":"2024-04-26T21:30:53.830821+0000","last_became_peered":"2024-04-26T21:30:53.830821+0000","last_unstale":"2024-04-26T21:30:55.136085+0000","last_undegraded":"2024-04-26T21:30:55.136085+0000","last_fullsized":"2024-04-26T21:30:55.136085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:08.540653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.300486+0000","last_change":"2024-04-26T21:30:52.812687+0000","last_active":"2024-04-26T21:30:57.300486+0000","last_peered":"2024-04-26T21:30:57.300486+0000","last_clean":"2024-04-26T21:30:57.300486+0000","last_became_active":"2024-04-26T21:30:52.812055+0000","last_became_peered":"2024-04-26T21:30:52.812055+0000","last_unstale":"2024-04-26T21:30:57.300486+0000","last_undegraded":"2024-04-26T21:30:57.300486+0000","last_fullsized":"2024-04-26T21:30:57.300486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:51.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133998+0000","last_change":"2024-04-26T21:30:53.829887+0000","last_active":"2024-04-26T21:30:55.133998+0000","last_peered":"2024-04-26T21:30:55.133998+0000","last_clean":"2024-04-26T21:30:55.133998+0000","last_became_active":"2024-04-26T21:30:53.829355+0000","last_became_peered":"2024-04-26T21:30:53.829355+0000","last_unstale":"2024-04-26T21:30:55.133998+0000","last_undegraded":"2024-04-26T21:30:55.133998+0000","last_fullsized":"2024-04-26T21:30:55.133998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:49.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134085+0000","last_change":"2024-04-26T21:30:52.808859+0000","last_active":"2024-04-26T21:30:55.134085+0000","last_peered":"2024-04-26T21:30:55.134085+0000","last_clean":"2024-04-26T21:30:55.134085+0000","last_became_active":"2024-04-26T21:30:52.807232+0000","last_became_peered":"2024-04-26T21:30:52.807232+0000","last_unstale":"2024-04-26T21:30:55.134085+0000","last_undegraded":"2024-04-26T21:30:55.134085+0000","last_fullsized":"2024-04-26T21:30:55.134085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:44.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.136231+0000","last_change":"2024-04-26T21:30:53.829723+0000","last_active":"2024-04-26T21:30:55.136231+0000","last_peered":"2024-04-26T21:30:55.136231+0000","last_clean":"2024-04-26T21:30:55.136231+0000","last_became_active":"2024-04-26T21:30:53.827799+0000","last_became_peered":"2024-04-26T21:30:53.827799+0000","last_unstale":"2024-04-26T21:30:55.136231+0000","last_undegraded":"2024-04-26T21:30:55.136231+0000","last_fullsized":"2024-04-26T21:30:55.136231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:20.798100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'17","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.889467+0000","last_change":"2024-04-26T21:30:52.812217+0000","last_active":"2024-04-26T21:31:00.889467+0000","last_peered":"2024-04-26T21:31:00.889467+0000","last_clean":"2024-04-26T21:31:00.889467+0000","last_became_active":"2024-04-26T21:30:52.811730+0000","last_became_peered":"2024-04-26T21:30:52.811730+0000","last_unstale":"2024-04-26T21:31:00.889467+0000","last_undegraded":"2024-04-26T21:31:00.889467+0000","last_fullsized":"2024-04-26T21:31:00.889467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:08.012161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":10,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965598+0000","last_change":"2024-04-26T21:30:53.813388+0000","last_active":"2024-04-26T21:30:54.965598+0000","last_peered":"2024-04-26T21:30:54.965598+0000","last_clean":"2024-04-26T21:30:54.965598+0000","last_became_active":"2024-04-26T21:30:53.812972+0000","last_became_peered":"2024-04-26T21:30:53.812972+0000","last_unstale":"2024-04-26T21:30:54.965598+0000","last_undegraded":"2024-04-26T21:30:54.965598+0000","last_fullsized":"2024-04-26T21:30:54.965598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:22.480641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.706119+0000","last_change":"2024-04-26T21:30:52.804426+0000","last_active":"2024-04-26T21:31:00.706119+0000","last_peered":"2024-04-26T21:31:00.706119+0000","last_clean":"2024-04-26T21:31:00.706119+0000","last_became_active":"2024-04-26T21:30:52.804083+0000","last_became_peered":"2024-04-26T21:30:52.804083+0000","last_unstale":"2024-04-26T21:31:00.706119+0000","last_undegraded":"2024-04-26T21:31:00.706119+0000","last_fullsized":"2024-04-26T21:31:00.706119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:10.164976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132691+0000","last_change":"2024-04-26T21:30:53.829326+0000","last_active":"2024-04-26T21:30:55.132691+0000","last_peered":"2024-04-26T21:30:55.132691+0000","last_clean":"2024-04-26T21:30:55.132691+0000","last_became_active":"2024-04-26T21:30:53.828863+0000","last_became_peered":"2024-04-26T21:30:53.828863+0000","last_unstale":"2024-04-26T21:30:55.132691+0000","last_undegraded":"2024-04-26T21:30:55.132691+0000","last_fullsized":"2024-04-26T21:30:55.132691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:56.756187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.298219+0000","last_change":"2024-04-26T21:30:52.805302+0000","last_active":"2024-04-26T21:30:57.298219+0000","last_peered":"2024-04-26T21:30:57.298219+0000","last_clean":"2024-04-26T21:30:57.298219+0000","last_became_active":"2024-04-26T21:30:52.804869+0000","last_became_peered":"2024-04-26T21:30:52.804869+0000","last_unstale":"2024-04-26T21:30:57.298219+0000","last_undegraded":"2024-04-26T21:30:57.298219+0000","last_fullsized":"2024-04-26T21:30:57.298219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:21.299474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974910+0000","last_change":"2024-04-26T21:30:53.823827+0000","last_active":"2024-04-26T21:30:54.974910+0000","last_peered":"2024-04-26T21:30:54.974910+0000","last_clean":"2024-04-26T21:30:54.974910+0000","last_became_active":"2024-04-26T21:30:53.823492+0000","last_became_peered":"2024-04-26T21:30:53.823492+0000","last_unstale":"2024-04-26T21:30:54.974910+0000","last_undegraded":"2024-04-26T21:30:54.974910+0000","last_fullsized":"2024-04-26T21:30:54.974910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.514753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.316576+0000","last_change":"2024-04-26T21:30:52.801468+0000","last_active":"2024-04-26T21:30:59.316576+0000","last_peered":"2024-04-26T21:30:59.316576+0000","last_clean":"2024-04-26T21:30:59.316576+0000","last_became_active":"2024-04-26T21:30:52.801071+0000","last_became_peered":"2024-04-26T21:30:52.801071+0000","last_unstale":"2024-04-26T21:30:59.316576+0000","last_undegraded":"2024-04-26T21:30:59.316576+0000","last_fullsized":"2024-04-26T21:30:59.316576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:52.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974518+0000","last_change":"2024-04-26T21:30:53.824075+0000","last_active":"2024-04-26T21:30:54.974518+0000","last_peered":"2024-04-26T21:30:54.974518+0000","last_clean":"2024-04-26T21:30:54.974518+0000","last_became_active":"2024-04-26T21:30:53.823753+0000","last_became_peered":"2024-04-26T21:30:53.823753+0000","last_unstale":"2024-04-26T21:30:54.974518+0000","last_undegraded":"2024-04-26T21:30:54.974518+0000","last_fullsized":"2024-04-26T21:30:54.974518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:44:09.294481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.342685+0000","last_change":"2024-04-26T21:30:52.806725+0000","last_active":"2024-04-26T21:31:00.342685+0000","last_peered":"2024-04-26T21:31:00.342685+0000","last_clean":"2024-04-26T21:31:00.342685+0000","last_became_active":"2024-04-26T21:30:52.806318+0000","last_became_peered":"2024-04-26T21:30:52.806318+0000","last_unstale":"2024-04-26T21:31:00.342685+0000","last_undegraded":"2024-04-26T21:31:00.342685+0000","last_fullsized":"2024-04-26T21:31:00.342685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:28.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133114+0000","last_change":"2024-04-26T21:30:53.835273+0000","last_active":"2024-04-26T21:30:55.133114+0000","last_peered":"2024-04-26T21:30:55.133114+0000","last_clean":"2024-04-26T21:30:55.133114+0000","last_became_active":"2024-04-26T21:30:53.834922+0000","last_became_peered":"2024-04-26T21:30:53.834922+0000","last_unstale":"2024-04-26T21:30:55.133114+0000","last_undegraded":"2024-04-26T21:30:55.133114+0000","last_fullsized":"2024-04-26T21:30:55.133114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:43.992579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.307685+0000","last_change":"2024-04-26T21:30:52.797639+0000","last_active":"2024-04-26T21:30:59.307685+0000","last_peered":"2024-04-26T21:30:59.307685+0000","last_clean":"2024-04-26T21:30:59.307685+0000","last_became_active":"2024-04-26T21:30:52.796576+0000","last_became_peered":"2024-04-26T21:30:52.796576+0000","last_unstale":"2024-04-26T21:30:59.307685+0000","last_undegraded":"2024-04-26T21:30:59.307685+0000","last_fullsized":"2024-04-26T21:30:59.307685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.730232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795853+0000","last_change":"2024-04-26T21:30:53.836190+0000","last_active":"2024-04-26T21:30:54.795853+0000","last_peered":"2024-04-26T21:30:54.795853+0000","last_clean":"2024-04-26T21:30:54.795853+0000","last_became_active":"2024-04-26T21:30:53.835302+0000","last_became_peered":"2024-04-26T21:30:53.835302+0000","last_unstale":"2024-04-26T21:30:54.795853+0000","last_undegraded":"2024-04-26T21:30:54.795853+0000","last_fullsized":"2024-04-26T21:30:54.795853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:13.222578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.206758+0000","last_change":"2024-04-26T21:30:52.809928+0000","last_active":"2024-04-26T21:30:55.206758+0000","last_peered":"2024-04-26T21:30:55.206758+0000","last_clean":"2024-04-26T21:30:55.206758+0000","last_became_active":"2024-04-26T21:30:52.808667+0000","last_became_peered":"2024-04-26T21:30:52.808667+0000","last_unstale":"2024-04-26T21:30:55.206758+0000","last_undegraded":"2024-04-26T21:30:55.206758+0000","last_fullsized":"2024-04-26T21:30:55.206758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:15.375105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:56.175776+0000","last_change":"2024-04-26T21:30:53.836649+0000","last_active":"2024-04-26T21:30:56.175776+0000","last_peered":"2024-04-26T21:30:56.175776+0000","last_clean":"2024-04-26T21:30:56.175776+0000","last_became_active":"2024-04-26T21:30:53.836252+0000","last_became_peered":"2024-04-26T21:30:53.836252+0000","last_unstale":"2024-04-26T21:30:56.175776+0000","last_undegraded":"2024-04-26T21:30:56.175776+0000","last_fullsized":"2024-04-26T21:30:56.175776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:53.797872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.222956+0000","last_change":"2024-04-26T21:30:52.808196+0000","last_active":"2024-04-26T21:30:55.222956+0000","last_peered":"2024-04-26T21:30:55.222956+0000","last_clean":"2024-04-26T21:30:55.222956+0000","last_became_active":"2024-04-26T21:30:52.806445+0000","last_became_peered":"2024-04-26T21:30:52.806445+0000","last_unstale":"2024-04-26T21:30:55.222956+0000","last_undegraded":"2024-04-26T21:30:55.222956+0000","last_fullsized":"2024-04-26T21:30:55.222956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:39.815485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.794891+0000","last_change":"2024-04-26T21:30:53.837998+0000","last_active":"2024-04-26T21:30:54.794891+0000","last_peered":"2024-04-26T21:30:54.794891+0000","last_clean":"2024-04-26T21:30:54.794891+0000","last_became_active":"2024-04-26T21:30:53.837443+0000","last_became_peered":"2024-04-26T21:30:53.837443+0000","last_unstale":"2024-04-26T21:30:54.794891+0000","last_undegraded":"2024-04-26T21:30:54.794891+0000","last_fullsized":"2024-04-26T21:30:54.794891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:04.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133492+0000","last_change":"2024-04-26T21:30:53.835501+0000","last_active":"2024-04-26T21:30:55.133492+0000","last_peered":"2024-04-26T21:30:55.133492+0000","last_clean":"2024-04-26T21:30:55.133492+0000","last_became_active":"2024-04-26T21:30:53.835096+0000","last_became_peered":"2024-04-26T21:30:53.835096+0000","last_unstale":"2024-04-26T21:30:55.133492+0000","last_undegraded":"2024-04-26T21:30:55.133492+0000","last_fullsized":"2024-04-26T21:30:55.133492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:26.178001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.316655+0000","last_change":"2024-04-26T21:30:52.805357+0000","last_active":"2024-04-26T21:30:58.316655+0000","last_peered":"2024-04-26T21:30:58.316655+0000","last_clean":"2024-04-26T21:30:58.316655+0000","last_became_active":"2024-04-26T21:30:52.804902+0000","last_became_peered":"2024-04-26T21:30:52.804902+0000","last_unstale":"2024-04-26T21:30:58.316655+0000","last_undegraded":"2024-04-26T21:30:58.316655+0000","last_fullsized":"2024-04-26T21:30:58.316655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:29.276141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856327+0000","last_change":"2024-04-26T21:30:53.835448+0000","last_active":"2024-04-26T21:30:54.856327+0000","last_peered":"2024-04-26T21:30:54.856327+0000","last_clean":"2024-04-26T21:30:54.856327+0000","last_became_active":"2024-04-26T21:30:53.835054+0000","last_became_peered":"2024-04-26T21:30:53.835054+0000","last_unstale":"2024-04-26T21:30:54.856327+0000","last_undegraded":"2024-04-26T21:30:54.856327+0000","last_fullsized":"2024-04-26T21:30:54.856327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:37.892713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.318992+0000","last_change":"2024-04-26T21:30:52.792942+0000","last_active":"2024-04-26T21:30:59.318992+0000","last_peered":"2024-04-26T21:30:59.318992+0000","last_clean":"2024-04-26T21:30:59.318992+0000","last_became_active":"2024-04-26T21:30:52.792553+0000","last_became_peered":"2024-04-26T21:30:52.792553+0000","last_unstale":"2024-04-26T21:30:59.318992+0000","last_undegraded":"2024-04-26T21:30:59.318992+0000","last_fullsized":"2024-04-26T21:30:59.318992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.915883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.974531+0000","last_change":"2024-04-26T21:29:15.217076+0000","last_active":"2024-04-26T21:30:54.974531+0000","last_peered":"2024-04-26T21:30:54.974531+0000","last_clean":"2024-04-26T21:30:54.974531+0000","last_became_active":"2024-04-26T21:29:15.215786+0000","last_became_peered":"2024-04-26T21:29:15.215786+0000","last_unstale":"2024-04-26T21:30:54.974531+0000","last_undegraded":"2024-04-26T21:30:54.974531+0000","last_fullsized":"2024-04-26T21:30:54.974531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_clean_scrub_stamp":"2024-04-26T21:24:37.649628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:47.795571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132321+0000","last_change":"2024-04-26T21:30:53.835344+0000","last_active":"2024-04-26T21:30:55.132321+0000","last_peered":"2024-04-26T21:30:55.132321+0000","last_clean":"2024-04-26T21:30:55.132321+0000","last_became_active":"2024-04-26T21:30:53.834838+0000","last_became_peered":"2024-04-26T21:30:53.834838+0000","last_unstale":"2024-04-26T21:30:55.132321+0000","last_undegraded":"2024-04-26T21:30:55.132321+0000","last_fullsized":"2024-04-26T21:30:55.132321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:37.588955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:58.317390+0000","last_change":"2024-04-26T21:30:52.808173+0000","last_active":"2024-04-26T21:30:58.317390+0000","last_peered":"2024-04-26T21:30:58.317390+0000","last_clean":"2024-04-26T21:30:58.317390+0000","last_became_active":"2024-04-26T21:30:52.807688+0000","last_became_peered":"2024-04-26T21:30:52.807688+0000","last_unstale":"2024-04-26T21:30:58.317390+0000","last_undegraded":"2024-04-26T21:30:58.317390+0000","last_fullsized":"2024-04-26T21:30:58.317390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:58.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.343730+0000","last_change":"2024-04-26T21:30:52.810654+0000","last_active":"2024-04-26T21:31:00.343730+0000","last_peered":"2024-04-26T21:31:00.343730+0000","last_clean":"2024-04-26T21:31:00.343730+0000","last_became_active":"2024-04-26T21:30:52.808959+0000","last_became_peered":"2024-04-26T21:30:52.808959+0000","last_unstale":"2024-04-26T21:31:00.343730+0000","last_undegraded":"2024-04-26T21:31:00.343730+0000","last_fullsized":"2024-04-26T21:31:00.343730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:59.134325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132507+0000","last_change":"2024-04-26T21:30:53.816961+0000","last_active":"2024-04-26T21:30:55.132507+0000","last_peered":"2024-04-26T21:30:55.132507+0000","last_clean":"2024-04-26T21:30:55.132507+0000","last_became_active":"2024-04-26T21:30:53.816571+0000","last_became_peered":"2024-04-26T21:30:53.816571+0000","last_unstale":"2024-04-26T21:30:55.132507+0000","last_undegraded":"2024-04-26T21:30:55.132507+0000","last_fullsized":"2024-04-26T21:30:55.132507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:43.218823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.328208+0000","last_change":"2024-04-26T21:30:52.801027+0000","last_active":"2024-04-26T21:30:59.328208+0000","last_peered":"2024-04-26T21:30:59.328208+0000","last_clean":"2024-04-26T21:30:59.328208+0000","last_became_active":"2024-04-26T21:30:52.797107+0000","last_became_peered":"2024-04-26T21:30:52.797107+0000","last_unstale":"2024-04-26T21:30:59.328208+0000","last_undegraded":"2024-04-26T21:30:59.328208+0000","last_fullsized":"2024-04-26T21:30:59.328208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:23.739673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856823+0000","last_change":"2024-04-26T21:30:53.821946+0000","last_active":"2024-04-26T21:30:54.856823+0000","last_peered":"2024-04-26T21:30:54.856823+0000","last_clean":"2024-04-26T21:30:54.856823+0000","last_became_active":"2024-04-26T21:30:53.821433+0000","last_became_peered":"2024-04-26T21:30:53.821433+0000","last_unstale":"2024-04-26T21:30:54.856823+0000","last_undegraded":"2024-04-26T21:30:54.856823+0000","last_fullsized":"2024-04-26T21:30:54.856823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:30.426334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.297454+0000","last_change":"2024-04-26T21:30:52.806302+0000","last_active":"2024-04-26T21:30:57.297454+0000","last_peered":"2024-04-26T21:30:57.297454+0000","last_clean":"2024-04-26T21:30:57.297454+0000","last_became_active":"2024-04-26T21:30:52.803402+0000","last_became_peered":"2024-04-26T21:30:52.803402+0000","last_unstale":"2024-04-26T21:30:57.297454+0000","last_undegraded":"2024-04-26T21:30:57.297454+0000","last_fullsized":"2024-04-26T21:30:57.297454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:07.163372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133762+0000","last_change":"2024-04-26T21:30:53.833803+0000","last_active":"2024-04-26T21:30:55.133762+0000","last_peered":"2024-04-26T21:30:55.133762+0000","last_clean":"2024-04-26T21:30:55.133762+0000","last_became_active":"2024-04-26T21:30:53.833378+0000","last_became_peered":"2024-04-26T21:30:53.833378+0000","last_unstale":"2024-04-26T21:30:55.133762+0000","last_undegraded":"2024-04-26T21:30:55.133762+0000","last_fullsized":"2024-04-26T21:30:55.133762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:58.505849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.296991+0000","last_change":"2024-04-26T21:30:52.810031+0000","last_active":"2024-04-26T21:30:57.296991+0000","last_peered":"2024-04-26T21:30:57.296991+0000","last_clean":"2024-04-26T21:30:57.296991+0000","last_became_active":"2024-04-26T21:30:52.809743+0000","last_became_peered":"2024-04-26T21:30:52.809743+0000","last_unstale":"2024-04-26T21:30:57.296991+0000","last_undegraded":"2024-04-26T21:30:57.296991+0000","last_fullsized":"2024-04-26T21:30:57.296991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:05.723934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.965099+0000","last_change":"2024-04-26T21:30:53.822563+0000","last_active":"2024-04-26T21:30:54.965099+0000","last_peered":"2024-04-26T21:30:54.965099+0000","last_clean":"2024-04-26T21:30:54.965099+0000","last_became_active":"2024-04-26T21:30:53.821848+0000","last_became_peered":"2024-04-26T21:30:53.821848+0000","last_unstale":"2024-04-26T21:30:54.965099+0000","last_undegraded":"2024-04-26T21:30:54.965099+0000","last_fullsized":"2024-04-26T21:30:54.965099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:39.861976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.296430+0000","last_change":"2024-04-26T21:30:52.792458+0000","last_active":"2024-04-26T21:30:57.296430+0000","last_peered":"2024-04-26T21:30:57.296430+0000","last_clean":"2024-04-26T21:30:57.296430+0000","last_became_active":"2024-04-26T21:30:52.792087+0000","last_became_peered":"2024-04-26T21:30:52.792087+0000","last_unstale":"2024-04-26T21:30:57.296430+0000","last_undegraded":"2024-04-26T21:30:57.296430+0000","last_fullsized":"2024-04-26T21:30:57.296430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:46.420509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795341+0000","last_change":"2024-04-26T21:30:53.826028+0000","last_active":"2024-04-26T21:30:54.795341+0000","last_peered":"2024-04-26T21:30:54.795341+0000","last_clean":"2024-04-26T21:30:54.795341+0000","last_became_active":"2024-04-26T21:30:53.822405+0000","last_became_peered":"2024-04-26T21:30:53.822405+0000","last_unstale":"2024-04-26T21:30:54.795341+0000","last_undegraded":"2024-04-26T21:30:54.795341+0000","last_fullsized":"2024-04-26T21:30:54.795341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:47.501074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:59.328237+0000","last_change":"2024-04-26T21:30:52.805708+0000","last_active":"2024-04-26T21:30:59.328237+0000","last_peered":"2024-04-26T21:30:59.328237+0000","last_clean":"2024-04-26T21:30:59.328237+0000","last_became_active":"2024-04-26T21:30:52.805258+0000","last_became_peered":"2024-04-26T21:30:52.805258+0000","last_unstale":"2024-04-26T21:30:59.328237+0000","last_undegraded":"2024-04-26T21:30:59.328237+0000","last_fullsized":"2024-04-26T21:30:59.328237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:58.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856322+0000","last_change":"2024-04-26T21:30:53.826016+0000","last_active":"2024-04-26T21:30:54.856322+0000","last_peered":"2024-04-26T21:30:54.856322+0000","last_clean":"2024-04-26T21:30:54.856322+0000","last_became_active":"2024-04-26T21:30:53.824050+0000","last_became_peered":"2024-04-26T21:30:53.824050+0000","last_unstale":"2024-04-26T21:30:54.856322+0000","last_undegraded":"2024-04-26T21:30:54.856322+0000","last_fullsized":"2024-04-26T21:30:54.856322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:35.816936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:57.301122+0000","last_change":"2024-04-26T21:30:52.811142+0000","last_active":"2024-04-26T21:30:57.301122+0000","last_peered":"2024-04-26T21:30:57.301122+0000","last_clean":"2024-04-26T21:30:57.301122+0000","last_became_active":"2024-04-26T21:30:52.810732+0000","last_became_peered":"2024-04-26T21:30:52.810732+0000","last_unstale":"2024-04-26T21:30:57.301122+0000","last_undegraded":"2024-04-26T21:30:57.301122+0000","last_fullsized":"2024-04-26T21:30:57.301122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:21.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975128+0000","last_change":"2024-04-26T21:30:53.832000+0000","last_active":"2024-04-26T21:30:54.975128+0000","last_peered":"2024-04-26T21:30:54.975128+0000","last_clean":"2024-04-26T21:30:54.975128+0000","last_became_active":"2024-04-26T21:30:53.831619+0000","last_became_peered":"2024-04-26T21:30:53.831619+0000","last_unstale":"2024-04-26T21:30:54.975128+0000","last_undegraded":"2024-04-26T21:30:54.975128+0000","last_fullsized":"2024-04-26T21:30:54.975128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:11.656424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:31:00.712535+0000","last_change":"2024-04-26T21:30:52.810311+0000","last_active":"2024-04-26T21:31:00.712535+0000","last_peered":"2024-04-26T21:31:00.712535+0000","last_clean":"2024-04-26T21:31:00.712535+0000","last_became_active":"2024-04-26T21:30:52.809976+0000","last_became_peered":"2024-04-26T21:30:52.809976+0000","last_unstale":"2024-04-26T21:31:00.712535+0000","last_undegraded":"2024-04-26T21:31:00.712535+0000","last_fullsized":"2024-04-26T21:31:00.712535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:49.146730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975123+0000","last_change":"2024-04-26T21:30:53.830315+0000","last_active":"2024-04-26T21:30:54.975123+0000","last_peered":"2024-04-26T21:30:54.975123+0000","last_clean":"2024-04-26T21:30:54.975123+0000","last_became_active":"2024-04-26T21:30:53.829439+0000","last_became_peered":"2024-04-26T21:30:53.829439+0000","last_unstale":"2024-04-26T21:30:54.975123+0000","last_undegraded":"2024-04-26T21:30:54.975123+0000","last_fullsized":"2024-04-26T21:30:54.975123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:49.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133896+0000","last_change":"2024-04-26T21:30:52.802392+0000","last_active":"2024-04-26T21:30:55.133896+0000","last_peered":"2024-04-26T21:30:55.133896+0000","last_clean":"2024-04-26T21:30:55.133896+0000","last_became_active":"2024-04-26T21:30:52.799978+0000","last_became_peered":"2024-04-26T21:30:52.799978+0000","last_unstale":"2024-04-26T21:30:55.133896+0000","last_undegraded":"2024-04-26T21:30:55.133896+0000","last_fullsized":"2024-04-26T21:30:55.133896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:45.993473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975106+0000","last_change":"2024-04-26T21:30:53.818118+0000","last_active":"2024-04-26T21:30:54.975106+0000","last_peered":"2024-04-26T21:30:54.975106+0000","last_clean":"2024-04-26T21:30:54.975106+0000","last_became_active":"2024-04-26T21:30:53.817712+0000","last_became_peered":"2024-04-26T21:30:53.817712+0000","last_unstale":"2024-04-26T21:30:54.975106+0000","last_undegraded":"2024-04-26T21:30:54.975106+0000","last_fullsized":"2024-04-26T21:30:54.975106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.819569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133542+0000","last_change":"2024-04-26T21:30:52.807308+0000","last_active":"2024-04-26T21:30:55.133542+0000","last_peered":"2024-04-26T21:30:55.133542+0000","last_clean":"2024-04-26T21:30:55.133542+0000","last_became_active":"2024-04-26T21:30:52.806935+0000","last_became_peered":"2024-04-26T21:30:52.806935+0000","last_unstale":"2024-04-26T21:30:55.133542+0000","last_undegraded":"2024-04-26T21:30:55.133542+0000","last_fullsized":"2024-04-26T21:30:55.133542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:42.549564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.975460+0000","last_change":"2024-04-26T21:30:53.812940+0000","last_active":"2024-04-26T21:30:54.975460+0000","last_peered":"2024-04-26T21:30:54.975460+0000","last_clean":"2024-04-26T21:30:54.975460+0000","last_became_active":"2024-04-26T21:30:53.812492+0000","last_became_peered":"2024-04-26T21:30:53.812492+0000","last_unstale":"2024-04-26T21:30:54.975460+0000","last_undegraded":"2024-04-26T21:30:54.975460+0000","last_fullsized":"2024-04-26T21:30:54.975460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:07.531226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795328+0000","last_change":"2024-04-26T21:30:53.822943+0000","last_active":"2024-04-26T21:30:54.795328+0000","last_peered":"2024-04-26T21:30:54.795328+0000","last_clean":"2024-04-26T21:30:54.795328+0000","last_became_active":"2024-04-26T21:30:53.822443+0000","last_became_peered":"2024-04-26T21:30:53.822443+0000","last_unstale":"2024-04-26T21:30:54.795328+0000","last_undegraded":"2024-04-26T21:30:54.795328+0000","last_fullsized":"2024-04-26T21:30:54.795328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:57.827470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133049+0000","last_change":"2024-04-26T21:30:53.832417+0000","last_active":"2024-04-26T21:30:55.133049+0000","last_peered":"2024-04-26T21:30:55.133049+0000","last_clean":"2024-04-26T21:30:55.133049+0000","last_became_active":"2024-04-26T21:30:53.832001+0000","last_became_peered":"2024-04-26T21:30:53.832001+0000","last_unstale":"2024-04-26T21:30:55.133049+0000","last_undegraded":"2024-04-26T21:30:55.133049+0000","last_fullsized":"2024-04-26T21:30:55.133049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:02.471766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134139+0000","last_change":"2024-04-26T21:30:53.817472+0000","last_active":"2024-04-26T21:30:55.134139+0000","last_peered":"2024-04-26T21:30:55.134139+0000","last_clean":"2024-04-26T21:30:55.134139+0000","last_became_active":"2024-04-26T21:30:53.817053+0000","last_became_peered":"2024-04-26T21:30:53.817053+0000","last_unstale":"2024-04-26T21:30:55.134139+0000","last_undegraded":"2024-04-26T21:30:55.134139+0000","last_fullsized":"2024-04-26T21:30:55.134139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:19.219132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133559+0000","last_change":"2024-04-26T21:30:53.826748+0000","last_active":"2024-04-26T21:30:55.133559+0000","last_peered":"2024-04-26T21:30:55.133559+0000","last_clean":"2024-04-26T21:30:55.133559+0000","last_became_active":"2024-04-26T21:30:53.826040+0000","last_became_peered":"2024-04-26T21:30:53.826040+0000","last_unstale":"2024-04-26T21:30:55.133559+0000","last_undegraded":"2024-04-26T21:30:55.133559+0000","last_fullsized":"2024-04-26T21:30:55.133559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:18.222414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132603+0000","last_change":"2024-04-26T21:30:53.825506+0000","last_active":"2024-04-26T21:30:55.132603+0000","last_peered":"2024-04-26T21:30:55.132603+0000","last_clean":"2024-04-26T21:30:55.132603+0000","last_became_active":"2024-04-26T21:30:53.824162+0000","last_became_peered":"2024-04-26T21:30:53.824162+0000","last_unstale":"2024-04-26T21:30:55.132603+0000","last_undegraded":"2024-04-26T21:30:55.132603+0000","last_fullsized":"2024-04-26T21:30:55.132603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:58.088580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795889+0000","last_change":"2024-04-26T21:30:53.825396+0000","last_active":"2024-04-26T21:30:54.795889+0000","last_peered":"2024-04-26T21:30:54.795889+0000","last_clean":"2024-04-26T21:30:54.795889+0000","last_became_active":"2024-04-26T21:30:53.825042+0000","last_became_peered":"2024-04-26T21:30:53.825042+0000","last_unstale":"2024-04-26T21:30:54.795889+0000","last_undegraded":"2024-04-26T21:30:54.795889+0000","last_fullsized":"2024-04-26T21:30:54.795889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:20.064265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.134033+0000","last_change":"2024-04-26T21:30:53.827819+0000","last_active":"2024-04-26T21:30:55.134033+0000","last_peered":"2024-04-26T21:30:55.134033+0000","last_clean":"2024-04-26T21:30:55.134033+0000","last_became_active":"2024-04-26T21:30:53.827027+0000","last_became_peered":"2024-04-26T21:30:53.827027+0000","last_unstale":"2024-04-26T21:30:55.134033+0000","last_undegraded":"2024-04-26T21:30:55.134033+0000","last_fullsized":"2024-04-26T21:30:55.134033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:49.586652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856701+0000","last_change":"2024-04-26T21:30:53.824693+0000","last_active":"2024-04-26T21:30:54.856701+0000","last_peered":"2024-04-26T21:30:54.856701+0000","last_clean":"2024-04-26T21:30:54.856701+0000","last_became_active":"2024-04-26T21:30:53.824278+0000","last_became_peered":"2024-04-26T21:30:53.824278+0000","last_unstale":"2024-04-26T21:30:54.856701+0000","last_undegraded":"2024-04-26T21:30:54.856701+0000","last_fullsized":"2024-04-26T21:30:54.856701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:42.799139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.133392+0000","last_change":"2024-04-26T21:30:53.823886+0000","last_active":"2024-04-26T21:30:55.133392+0000","last_peered":"2024-04-26T21:30:55.133392+0000","last_clean":"2024-04-26T21:30:55.133392+0000","last_became_active":"2024-04-26T21:30:53.823443+0000","last_became_peered":"2024-04-26T21:30:53.823443+0000","last_unstale":"2024-04-26T21:30:55.133392+0000","last_undegraded":"2024-04-26T21:30:55.133392+0000","last_fullsized":"2024-04-26T21:30:55.133392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.022073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.856151+0000","last_change":"2024-04-26T21:30:53.825329+0000","last_active":"2024-04-26T21:30:54.856151+0000","last_peered":"2024-04-26T21:30:54.856151+0000","last_clean":"2024-04-26T21:30:54.856151+0000","last_became_active":"2024-04-26T21:30:53.822833+0000","last_became_peered":"2024-04-26T21:30:53.822833+0000","last_unstale":"2024-04-26T21:30:54.856151+0000","last_undegraded":"2024-04-26T21:30:54.856151+0000","last_fullsized":"2024-04-26T21:30:54.856151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.840177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.135530+0000","last_change":"2024-04-26T21:30:53.837657+0000","last_active":"2024-04-26T21:30:55.135530+0000","last_peered":"2024-04-26T21:30:55.135530+0000","last_clean":"2024-04-26T21:30:55.135530+0000","last_became_active":"2024-04-26T21:30:53.837354+0000","last_became_peered":"2024-04-26T21:30:53.837354+0000","last_unstale":"2024-04-26T21:30:55.135530+0000","last_undegraded":"2024-04-26T21:30:55.135530+0000","last_fullsized":"2024-04-26T21:30:55.135530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:34.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:55.132680+0000","last_change":"2024-04-26T21:30:53.811372+0000","last_active":"2024-04-26T21:30:55.132680+0000","last_peered":"2024-04-26T21:30:55.132680+0000","last_clean":"2024-04-26T21:30:55.132680+0000","last_became_active":"2024-04-26T21:30:53.810124+0000","last_became_peered":"2024-04-26T21:30:53.810124+0000","last_unstale":"2024-04-26T21:30:55.132680+0000","last_undegraded":"2024-04-26T21:30:55.132680+0000","last_fullsized":"2024-04-26T21:30:55.132680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:14.134041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T21:30:54.795371+0000","last_change":"2024-04-26T21:30:53.836391+0000","last_active":"2024-04-26T21:30:54.795371+0000","last_peered":"2024-04-26T21:30:54.795371+0000","last_clean":"2024-04-26T21:30:54.795371+0000","last_became_active":"2024-04-26T21:30:53.835987+0000","last_became_peered":"2024-04-26T21:30:53.835987+0000","last_unstale":"2024-04-26T21:30:54.795371+0000","last_undegraded":"2024-04-26T21:30:54.795371+0000","last_fullsized":"2024-04-26T21:30:54.795371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:56.449077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26302,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":184,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":58423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547224,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1045929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710751,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":452764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972391,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":453356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266733,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1044688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":451856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":475505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":452050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":452522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1552384,"data_stored":1042406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:31:04.324 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:31:04.325 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T21:31:04.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T21:31:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:05.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:04 smithi143 ceph-mon[31461]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:05.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:04 smithi097 ceph-mon[28127]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:04 smithi111 ceph-mon[31001]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:05 smithi143 ceph-mon[31461]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:05 smithi143 ceph-mon[31461]: pgmap v370: 97 pgs: 97 active+clean; 603 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8.3 KiB/s wr, 23 op/s 2024-04-26T21:31:06.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:05 smithi097 ceph-mon[28127]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:06.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:05 smithi097 ceph-mon[28127]: pgmap v370: 97 pgs: 97 active+clean; 603 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8.3 KiB/s wr, 23 op/s 2024-04-26T21:31:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:05 smithi111 ceph-mon[31001]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:31:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:05 smithi111 ceph-mon[31001]: pgmap v370: 97 pgs: 97 active+clean; 603 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8.3 KiB/s wr, 23 op/s 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 -- 172.21.15.97:0/1962702955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c075e80 msgr2=0x7fcc3c076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 --2- 172.21.15.97:0/1962702955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c075e80 0x7fcc3c076300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400b0a0 tx=0x7fcc2400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 -- 172.21.15.97:0/1962702955 shutdown_connections 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 --2- 172.21.15.97:0/1962702955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c111e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 --2- 172.21.15.97:0/1962702955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 0x7fcc3c0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 --2- 172.21.15.97:0/1962702955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c075e80 0x7fcc3c076300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 -- 172.21.15.97:0/1962702955 >> 172.21.15.97:0/1962702955 conn(0x7fcc3c0fea40 msgr2=0x7fcc3c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 -- 172.21.15.97:0/1962702955 shutdown_connections 2024-04-26T21:31:06.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.993+0000 7fcc42800700 1 -- 172.21.15.97:0/1962702955 wait complete. 2024-04-26T21:31:06.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.994+0000 7fcc42800700 1 Processor -- start 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 -- start start 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c075e80 0x7fcc3c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 0x7fcc3c19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c115430 con 0x7fcc3c10fa80 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1152b0 con 0x7fcc3c075e80 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc42800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1155b0 con 0x7fcc3c077450 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36154/0 (socket says 172.21.15.97:36154) 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc3be00700 1 -- 172.21.15.97:0/278884222 learned_addr learned my addr 172.21.15.97:0/278884222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.995+0000 7fcc3b400700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c075e80 0x7fcc3c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc32a00700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 0x7fcc3c19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 -- 172.21.15.97:0/278884222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 msgr2=0x7fcc3c19ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 0x7fcc3c19ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 -- 172.21.15.97:0/278884222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c075e80 msgr2=0x7fcc3c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c075e80 0x7fcc3c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 -- 172.21.15.97:0/278884222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c1a76d0 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.996+0000 7fcc3be00700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c930 tx=0x7fcc3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.997+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340078d0 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.997+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc3400f450 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.997+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c1a79c0 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.997+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34018660 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.997+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1a7f00 con 0x7fcc3c10fa80 2024-04-26T21:31:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:06.998+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c051960 con 0x7fcc3c10fa80 2024-04-26T21:31:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.000+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc34007a70 con 0x7fcc3c10fa80 2024-04-26T21:31:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.001+0000 7fcc38c00700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 0x7fcc200a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.001+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcc340d02b0 con 0x7fcc3c10fa80 2024-04-26T21:31:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.002+0000 7fcc3b400700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 0x7fcc200a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:07.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.002+0000 7fcc3b400700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 0x7fcc200a6240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005950 tx=0x7fcc2c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.006+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcc34014030 con 0x7fcc3c10fa80 2024-04-26T21:31:07.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.323+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fcc3c02e510 con 0x7fcc3c10fa80 2024-04-26T21:31:07.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.327+0000 7fcc38c00700 1 -- 172.21.15.97:0/278884222 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fcc3409cbe0 con 0x7fcc3c10fa80 2024-04-26T21:31:07.328 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-26T21:31:07.328 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAbHSxmI6JlExAAO9auZuFIhhdbOHv7QrURpw== 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.330+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 msgr2=0x7fcc200a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.330+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 0x7fcc200a6240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005950 tx=0x7fcc2c00a300 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 msgr2=0x7fcc3c1a7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c930 tx=0x7fcc3400cc40 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 shutdown_connections 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcc200a3d80 0x7fcc200a6240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc3c075e80 0x7fcc3c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc3c10fa80 0x7fcc3c1a7030 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 --2- 172.21.15.97:0/278884222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc3c077450 0x7fcc3c19ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 >> 172.21.15.97:0/278884222 conn(0x7fcc3c0fea40 msgr2=0x7fcc3c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:07.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 shutdown_connections 2024-04-26T21:31:07.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:07.331+0000 7fcc42800700 1 -- 172.21.15.97:0/278884222 wait complete. 2024-04-26T21:31:08.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:07 smithi143 ceph-mon[31461]: pgmap v371: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-04-26T21:31:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:31:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:31:08.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:07 smithi097 ceph-mon[28127]: pgmap v371: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-04-26T21:31:08.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:31:08.048 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:31:08.133 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:31:08.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T21:31:08.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:31:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:07 smithi111 ceph-mon[31001]: pgmap v371: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-04-26T21:31:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:31:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/278884222' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:31:08.264 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:31:08.264 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:31:08.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T21:31:08.762 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:09.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:09 smithi097 ceph-mon[28127]: pgmap v372: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-04-26T21:31:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:09 smithi143 ceph-mon[31461]: pgmap v372: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-04-26T21:31:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:09 smithi111 ceph-mon[31001]: pgmap v372: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 -- 172.21.15.97:0/3316011943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c410b640 msgr2=0x7fd0c410ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 --2- 172.21.15.97:0/3316011943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c410b640 0x7fd0c410ba20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009990 tx=0x7fd0b8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 -- 172.21.15.97:0/3316011943 shutdown_connections 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 --2- 172.21.15.97:0/3316011943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4106000 0x7fd0c4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 --2- 172.21.15.97:0/3316011943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c410b640 0x7fd0c410ba20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 --2- 172.21.15.97:0/3316011943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c4105640 0x7fd0c4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.981+0000 7fd0cc400700 1 -- 172.21.15.97:0/3316011943 >> 172.21.15.97:0/3316011943 conn(0x7fd0c4101130 msgr2=0x7fd0c4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.982+0000 7fd0cc400700 1 -- 172.21.15.97:0/3316011943 shutdown_connections 2024-04-26T21:31:10.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.982+0000 7fd0cc400700 1 -- 172.21.15.97:0/3316011943 wait complete. 2024-04-26T21:31:10.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.983+0000 7fd0cc400700 1 Processor -- start 2024-04-26T21:31:10.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.983+0000 7fd0cc400700 1 -- start start 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4105640 0x7fd0c41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c410b640 0x7fd0c419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41156d0 con 0x7fd0c4106000 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4115550 con 0x7fd0c410b640 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0cc400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4115850 con 0x7fd0c4105640 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36508/0 (socket says 172.21.15.97:36508) 2024-04-26T21:31:10.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 -- 172.21.15.97:0/2751859882 learned_addr learned my addr 172.21.15.97:0/2751859882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:10.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9a00700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4105640 0x7fd0c41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:10.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0ca400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c410b640 0x7fd0c419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:10.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 -- 172.21.15.97:0/2751859882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4105640 msgr2=0x7fd0c41a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:10.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4105640 0x7fd0c41a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 -- 172.21.15.97:0/2751859882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c410b640 msgr2=0x7fd0c419ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c410b640 0x7fd0c419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.984+0000 7fd0c9000700 1 -- 172.21.15.97:0/2751859882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c419c190 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0c9000700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000d980 tx=0x7fd0b000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0016070 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b0004020 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c419c480 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004bb0 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.985+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c41a8740 con 0x7fd0c4106000 2024-04-26T21:31:10.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.986+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c406a110 con 0x7fd0c4106000 2024-04-26T21:31:10.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.988+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd0b000bb60 con 0x7fd0c4106000 2024-04-26T21:31:10.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.990+0000 7fd0c2000700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 0x7fd0ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:10.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.990+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd0b00d0310 con 0x7fd0c4106000 2024-04-26T21:31:10.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.990+0000 7fd0c9a00700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 0x7fd0ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:10.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.991+0000 7fd0c9a00700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 0x7fd0ac0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800b5c0 tx=0x7fd0b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:10.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:10.996+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd0b001f020 con 0x7fd0c4106000 2024-04-26T21:31:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.283+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd0c402e490 con 0x7fd0c4106000 2024-04-26T21:31:11.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.298+0000 7fd0c2000700 1 -- 172.21.15.97:0/2751859882 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7fd0b000f3c0 con 0x7fd0c4106000 2024-04-26T21:31:11.301 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 msgr2=0x7fd0ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 0x7fd0ac0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800b5c0 tx=0x7fd0b8005fb0 comp rx=0 tx=0).stop 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 msgr2=0x7fd0c41a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000d980 tx=0x7fd0b000dd40 comp rx=0 tx=0).stop 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 shutdown_connections 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd0ac0a3cd0 0x7fd0ac0a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c410b640 0x7fd0c419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c4105640 0x7fd0c41a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:11.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 --2- 172.21.15.97:0/2751859882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4106000 0x7fd0c41a1db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:11.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 >> 172.21.15.97:0/2751859882 conn(0x7fd0c4101130 msgr2=0x7fd0c41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:11.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 shutdown_connections 2024-04-26T21:31:11.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:11.306+0000 7fd0cc400700 1 -- 172.21.15.97:0/2751859882 wait complete. 2024-04-26T21:31:12.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:11 smithi143 ceph-mon[31461]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-26T21:31:12.074 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:11 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2751859882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:31:12.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:11 smithi097 ceph-mon[28127]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-26T21:31:12.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:11 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2751859882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:31:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:11 smithi111 ceph-mon[31001]: pgmap v373: 97 pgs: 97 active+clean; 627 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-26T21:31:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:11 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2751859882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:31:12.163 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3172837618},{"type":"v1","addr":"172.21.15.97:6801","nonce":3172837618}]},"active_addr":"172.21.15.97:6801/3172837618","active_change":"2024-04-26T21:21:20.482625+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3841450061}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4292485846}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1008162826}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3778554}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":861049036}]}]} 2024-04-26T21:31:12.168 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T21:31:12.168 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T21:31:12.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:31:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:13.870 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:13 smithi097 ceph-mon[28127]: pgmap v374: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-04-26T21:31:13.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:13 smithi111 ceph-mon[31001]: pgmap v374: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-04-26T21:31:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:13 smithi143 ceph-mon[31461]: pgmap v374: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2024-04-26T21:31:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 -- 172.21.15.97:0/3452968364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406c680 msgr2=0x7f1e14105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 --2- 172.21.15.97:0/3452968364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406c680 0x7f1e14105e80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b0a0 tx=0x7f1e0c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 -- 172.21.15.97:0/3452968364 shutdown_connections 2024-04-26T21:31:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 --2- 172.21.15.97:0/3452968364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e14108370 0x7f1e14108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 --2- 172.21.15.97:0/3452968364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406c680 0x7f1e14105e80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 --2- 172.21.15.97:0/3452968364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e1406baf0 0x7f1e1406bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 -- 172.21.15.97:0/3452968364 >> 172.21.15.97:0/3452968364 conn(0x7f1e140792d0 msgr2=0x7f1e140796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 -- 172.21.15.97:0/3452968364 shutdown_connections 2024-04-26T21:31:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.788+0000 7f1e1c000700 1 -- 172.21.15.97:0/3452968364 wait complete. 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.790+0000 7f1e1c000700 1 Processor -- start 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.790+0000 7f1e1c000700 1 -- start start 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 0x7f1e141a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 0x7f1e141a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14119ab0 con 0x7f1e1406baf0 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14119930 con 0x7f1e1406c680 2024-04-26T21:31:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e1c000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14119c30 con 0x7f1e14108370 2024-04-26T21:31:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e19600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e19600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36532/0 (socket says 172.21.15.97:36532) 2024-04-26T21:31:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e19600700 1 -- 172.21.15.97:0/353549249 learned_addr learned my addr 172.21.15.97:0/353549249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.791+0000 7f1e18c00700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 0x7f1e141a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e1a000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 0x7f1e141a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 -- 172.21.15.97:0/353549249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 msgr2=0x7f1e141a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 0x7f1e141a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 -- 172.21.15.97:0/353549249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 msgr2=0x7f1e141a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 0x7f1e141a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 -- 172.21.15.97:0/353549249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e141a4f70 con 0x7f1e1406baf0 2024-04-26T21:31:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e1a000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 0x7f1e141a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e18c00700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 0x7f1e141a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.792+0000 7f1e19600700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c9b0 tx=0x7f1e0800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.793+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08007b00 con 0x7f1e1406baf0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.793+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e08007ca0 con 0x7f1e1406baf0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.793+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e141a5260 con 0x7f1e1406baf0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.793+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08018660 con 0x7f1e1406baf0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.793+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e141acae0 con 0x7f1e1406baf0 2024-04-26T21:31:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.794+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e14102960 con 0x7f1e1406baf0 2024-04-26T21:31:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.796+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e0800f450 con 0x7f1e1406baf0 2024-04-26T21:31:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.798+0000 7f1e12000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 0x7f1dfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.798+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1e080d0530 con 0x7f1e1406baf0 2024-04-26T21:31:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.798+0000 7f1e18c00700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 0x7f1dfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:14.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.799+0000 7f1e18c00700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 0x7f1dfc0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e14079af0 tx=0x7f1e00005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:14.803+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1e08014030 con 0x7f1e1406baf0 2024-04-26T21:31:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.030+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e141acd10 con 0x7f1e1406baf0 2024-04-26T21:31:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.035+0000 7f1e12000700 1 -- 172.21.15.97:0/353549249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22220 (secure 0 0 0) 0x7f1e0809ce60 con 0x7f1e1406baf0 2024-04-26T21:31:15.036 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:15.036 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:30:54.788801+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:31:15.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 msgr2=0x7f1dfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 0x7f1dfc0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e14079af0 tx=0x7f1e00005f30 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 msgr2=0x7f1e1419ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c9b0 tx=0x7f1e0800cd70 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 shutdown_connections 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1dfc0a3cd0 0x7f1dfc0a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e14108370 0x7f1e141a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e1406baf0 0x7f1e1419ff60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 --2- 172.21.15.97:0/353549249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e1406c680 0x7f1e141a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.038+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 >> 172.21.15.97:0/353549249 conn(0x7f1e140792d0 msgr2=0x7f1e141016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.039+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 shutdown_connections 2024-04-26T21:31:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:15.039+0000 7f1e1c000700 1 -- 172.21.15.97:0/353549249 wait complete. 2024-04-26T21:31:15.752 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T21:31:15.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:31:16.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:15 smithi143 ceph-mon[31461]: pgmap v375: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-26T21:31:16.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:15 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/353549249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:16.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:15 smithi097 ceph-mon[28127]: pgmap v375: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-26T21:31:16.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:15 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/353549249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:15 smithi111 ceph-mon[31001]: pgmap v375: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-26T21:31:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:15 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/353549249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:16.206 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:18.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:17 smithi097 ceph-mon[28127]: pgmap v376: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-04-26T21:31:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:17 smithi143 ceph-mon[31461]: pgmap v376: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-04-26T21:31:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:17 smithi111 ceph-mon[31001]: pgmap v376: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 -- 172.21.15.97:0/2874696577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 msgr2=0x7fa2801098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 --2- 172.21.15.97:0/2874696577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2801098a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa274009990 tx=0x7fa274009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 -- 172.21.15.97:0/2874696577 shutdown_connections 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 --2- 172.21.15.97:0/2874696577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 0x7fa280110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 --2- 172.21.15.97:0/2874696577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2801098a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 --2- 172.21.15.97:0/2874696577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 0x7fa2801038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 -- 172.21.15.97:0/2874696577 >> 172.21.15.97:0/2874696577 conn(0x7fa28007a580 msgr2=0x7fa28007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 -- 172.21.15.97:0/2874696577 shutdown_connections 2024-04-26T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.335+0000 7fa288400700 1 -- 172.21.15.97:0/2874696577 wait complete. 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.336+0000 7fa288400700 1 Processor -- start 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.336+0000 7fa288400700 1 -- start start 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 0x7fa280075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 0x7fa280075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa280113540 con 0x7fa2801094c0 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801133c0 con 0x7fa280103470 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa288400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801136c0 con 0x7fa280103e30 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa286400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa286400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36546/0 (socket says 172.21.15.97:36546) 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa286400700 1 -- 172.21.15.97:0/3426181851 learned_addr learned my addr 172.21.15.97:0/3426181851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.337+0000 7fa285a00700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 0x7fa280075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa285000700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 0x7fa280075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 -- 172.21.15.97:0/3426181851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 msgr2=0x7fa280075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 0x7fa280075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 -- 172.21.15.97:0/3426181851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 msgr2=0x7fa280075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 0x7fa280075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 -- 172.21.15.97:0/3426181851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2801a7fa0 con 0x7fa2801094c0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa286400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b970 tx=0x7fa27800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278019070 con 0x7fa2801094c0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2801a8290 con 0x7fa2801094c0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27800cdc0 con 0x7fa2801094c0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.338+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278014e30 con 0x7fa2801094c0 2024-04-26T21:31:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.339+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a8780 con 0x7fa2801094c0 2024-04-26T21:31:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.340+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28006a110 con 0x7fa2801094c0 2024-04-26T21:31:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.342+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa27801c070 con 0x7fa2801094c0 2024-04-26T21:31:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.343+0000 7fa27e000700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 0x7fa2680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.343+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa2780cf710 con 0x7fa2801094c0 2024-04-26T21:31:18.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.343+0000 7fa285a00700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 0x7fa2680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.344+0000 7fa285a00700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 0x7fa2680a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa274003760 tx=0x7fa27400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:18.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.348+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa27809c150 con 0x7fa2801094c0 2024-04-26T21:31:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.571+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa28002e490 con 0x7fa2801094c0 2024-04-26T21:31:18.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.573+0000 7fa27e000700 1 -- 172.21.15.97:0/3426181851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22220 (secure 0 0 0) 0x7fa27809ba50 con 0x7fa2801094c0 2024-04-26T21:31:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:31:18.575 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:30:54.788801+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:31:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 msgr2=0x7fa2680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 0x7fa2680a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa274003760 tx=0x7fa27400b540 comp rx=0 tx=0).stop 2024-04-26T21:31:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 msgr2=0x7fa2800761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b970 tx=0x7fa27800bc80 comp rx=0 tx=0).stop 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 shutdown_connections 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa2680a3cd0 0x7fa2680a6190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa280103470 0x7fa280075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2801094c0 0x7fa2800761a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 --2- 172.21.15.97:0/3426181851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa280103e30 0x7fa280075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 >> 172.21.15.97:0/3426181851 conn(0x7fa28007a580 msgr2=0x7fa280102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 shutdown_connections 2024-04-26T21:31:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:18.576+0000 7fa288400700 1 -- 172.21.15.97:0/3426181851 wait complete. 2024-04-26T21:31:18.942 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:18 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3426181851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:19.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:18 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3426181851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:19.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:18 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3426181851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:31:19.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T21:31:19.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T21:31:19.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T21:31:19.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T21:31:19.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T21:31:19.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T21:31:19.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T21:31:19.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T21:31:19.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T21:31:19.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T21:31:19.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T21:31:19.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T21:31:20.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:19 smithi097 ceph-mon[28127]: pgmap v377: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:19 smithi111 ceph-mon[31001]: pgmap v377: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:19 smithi143 ceph-mon[31461]: pgmap v377: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:20.407 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.452 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.511 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:20.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:31:20.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T21:31:20.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T21:31:20.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:31:20.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:20 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-26T21:31:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:31:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T21:31:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T21:31:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:31:21.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:20 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-26T21:31:21.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:31:21.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T21:31:21.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T21:31:21.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:31:21.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:20 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-26T21:31:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: pgmap v378: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:22.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:31:22.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T21:31:22.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T21:31:22.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:31:22.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-26T21:31:22.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:21 smithi097 ceph-mon[28127]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:31:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: pgmap v378: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:31:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T21:31:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T21:31:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:31:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-26T21:31:22.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:21 smithi111 ceph-mon[31001]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: pgmap v378: 97 pgs: 97 active+clean; 634 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-26T21:31:22.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:21 smithi143 ceph-mon[31461]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:31:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:22 smithi111 ceph-mon[31001]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:31:23.183 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:22 smithi097 ceph-mon[28127]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:31:23.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:22 smithi143 ceph-mon[31461]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:31:23.941 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:23 smithi097 ceph-mon[28127]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-26T21:31:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:23 smithi111 ceph-mon[31001]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-26T21:31:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:23 smithi143 ceph-mon[31461]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-26T21:31:25.212 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:24 smithi097 ceph-mon[28127]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:25.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:24 smithi143 ceph-mon[31461]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:25.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:24 smithi111 ceph-mon[31001]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:25.456 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:25.458 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:25.587 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:25.588 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:25.589 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:28.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:27 smithi143 ceph-mon[31461]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:28.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:27 smithi097 ceph-mon[28127]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:27 smithi111 ceph-mon[31001]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:29.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:29 smithi097 ceph-mon[28127]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:30.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:29 smithi143 ceph-mon[31461]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:29 smithi111 ceph-mon[31001]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T21:31:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 -- 172.21.15.97:0/2001678896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806b130 msgr2=0x7f916806b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 --2- 172.21.15.97:0/2001678896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806b130 0x7f916806b590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9154009990 tx=0x7f9154009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 -- 172.21.15.97:0/2001678896 shutdown_connections 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 --2- 172.21.15.97:0/2001678896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 0x7f9168109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 --2- 172.21.15.97:0/2001678896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916806bad0 0x7f91681102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 --2- 172.21.15.97:0/2001678896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806b130 0x7f916806b590 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.275+0000 7f916de00700 1 -- 172.21.15.97:0/2001678896 >> 172.21.15.97:0/2001678896 conn(0x7f91680792d0 msgr2=0x7f91680796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.276+0000 7f916de00700 1 -- 172.21.15.97:0/2001678896 shutdown_connections 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.276+0000 7f916de00700 1 -- 172.21.15.97:0/2001678896 wait complete. 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.276+0000 7f916de00700 1 Processor -- start 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 -- start start 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916806b130 0x7f916819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 0x7f916819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 0x7f91681a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9168113590 con 0x7f916806bad0 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9168113410 con 0x7f91681094a0 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.277+0000 7f916de00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9168113710 con 0x7f916806b130 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 0x7f916819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 0x7f916819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55940/0 (socket says 172.21.15.97:55940) 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 -- 172.21.15.97:0/3932944214 learned_addr learned my addr 172.21.15.97:0/3932944214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166a00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916806b130 0x7f916819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9167400700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 0x7f91681a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 -- 172.21.15.97:0/3932944214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916806b130 msgr2=0x7f916819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916806b130 0x7f916819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 -- 172.21.15.97:0/3932944214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 msgr2=0x7f91681a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 0x7f91681a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9166000700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91681a0df0 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.278+0000 7f9167400700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91681094a0 0x7f91681a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f9166000700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 0x7f916819c320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f91540057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154027070 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91681a1080 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a8960 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f915400fc00 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.279+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915402b980 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.280+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9148001030 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.283+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f915402a070 con 0x7f916806bad0 2024-04-26T21:31:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f916ca00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f914c0a3d50 0x7f914c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f91540d12a0 con 0x7f916806bad0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f9166a00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f914c0a3d50 0x7f914c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f916ca00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] conn(0x7f914c0ad710 0x7f914c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f914c0b0280 con 0x7f914c0ad710 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.284+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f91540d1650 con 0x7f916806bad0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.285+0000 7f9167400700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] conn(0x7f914c0ad710 0x7f914c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.285+0000 7f9166a00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f914c0a3d50 0x7f914c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9158009a10 tx=0x7f9158009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.285+0000 7f9167400700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] conn(0x7f914c0ad710 0x7f914c0afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.286+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== osd.0 v2:172.21.15.97:6802/3890416768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f914c0b0280 con 0x7f914c0ad710 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.305+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 --> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9148002e60 con 0x7f914c0ad710 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.305+0000 7f916ca00700 1 -- 172.21.15.97:0/3932944214 <== osd.0 v2:172.21.15.97:6802/3890416768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9148002e60 con 0x7f914c0ad710 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] conn(0x7f914c0ad710 msgr2=0x7f914c0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] conn(0x7f914c0ad710 0x7f914c0afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f914c0a3d50 msgr2=0x7f914c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f914c0a3d50 0x7f914c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9158009a10 tx=0x7f9158009190 comp rx=0 tx=0).stop 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 msgr2=0x7f916819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 --2- 172.21.15.97:0/3932944214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916806bad0 0x7f916819c320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f91540057c0 comp rx=0 tx=0).stop 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f9167400700 1 -- 172.21.15.97:0/3932944214 reap_dead start 2024-04-26T21:31:30.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 shutdown_connections 2024-04-26T21:31:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.306+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 >> 172.21.15.97:0/3932944214 conn(0x7f91680792d0 msgr2=0x7f916810ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.307+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 shutdown_connections 2024-04-26T21:31:30.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:28.307+0000 7f916de00700 1 -- 172.21.15.97:0/3932944214 wait complete. 2024-04-26T21:31:31.735 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:31 smithi097 ceph-mon[28127]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 311 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T21:31:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:31 smithi143 ceph-mon[31461]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 311 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T21:31:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:31 smithi111 ceph-mon[31001]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 311 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 -- 172.21.15.97:0/1896032098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c108690 msgr2=0x7f9b3c110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 --2- 172.21.15.97:0/1896032098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c108690 0x7f9b3c110260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b0d0 tx=0x7f9b3400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 -- 172.21.15.97:0/1896032098 shutdown_connections 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 --2- 172.21.15.97:0/1896032098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c108690 0x7f9b3c110260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 --2- 172.21.15.97:0/1896032098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 --2- 172.21.15.97:0/1896032098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c103f60 0x7f9b3c107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.011+0000 7f9b41a00700 1 -- 172.21.15.97:0/1896032098 >> 172.21.15.97:0/1896032098 conn(0x7f9b3c0feea0 msgr2=0x7f9b3c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.012+0000 7f9b41a00700 1 -- 172.21.15.97:0/1896032098 shutdown_connections 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.012+0000 7f9b41a00700 1 -- 172.21.15.97:0/1896032098 wait complete. 2024-04-26T21:31:33.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.013+0000 7f9b41a00700 1 Processor -- start 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.013+0000 7f9b41a00700 1 -- start start 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b3c103f60 0x7f9b3c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c108690 0x7f9b3c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c113450 con 0x7f9b3c1035b0 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1132d0 con 0x7f9b3c108690 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1135d0 con 0x7f9b3c103f60 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48840/0 (socket says 172.21.15.97:48840) 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b3be00700 1 -- 172.21.15.97:0/3844434192 learned_addr learned my addr 172.21.15.97:0/3844434192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b41000700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c108690 0x7f9b3c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b3be00700 1 -- 172.21.15.97:0/3844434192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b3c103f60 msgr2=0x7f9b3c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.014+0000 7f9b3be00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b3c103f60 0x7f9b3c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b3be00700 1 -- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c108690 msgr2=0x7f9b3c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b3be00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b3c108690 0x7f9b3c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b3be00700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c199fb0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b3be00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c19f600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c970 tx=0x7f9b2c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c007aa0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b2c004cc0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c19a2a0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c005500 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.015+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c1a6460 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.016+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b3c063bb0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.018+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b2c007c40 con 0x7f9b3c1035b0 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.019+0000 7f9b39600700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9b200a3cd0 0x7f9b200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.020+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9b2c0d0840 con 0x7f9b3c1035b0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.020+0000 7f9b33e00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9b200a3cd0 0x7f9b200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.020+0000 7f9b39600700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] conn(0x7f9b200ad690 0x7f9b200afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.020+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b2c00eec0 con 0x7f9b200ad690 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.020+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9b2c0d0bf0 con 0x7f9b3c1035b0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.021+0000 7f9b41000700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] conn(0x7f9b200ad690 0x7f9b200afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.021+0000 7f9b33e00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9b200a3cd0 0x7f9b200a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c109c30 tx=0x7f9b24005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.022+0000 7f9b41000700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] conn(0x7f9b200ad690 0x7f9b200afb30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.024+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== osd.7 v2:172.21.15.111:6816/3525126129 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b2c00eec0 con 0x7f9b200ad690 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.042+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 --> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b3c064030 con 0x7f9b200ad690 2024-04-26T21:31:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.043+0000 7f9b39600700 1 -- 172.21.15.97:0/3844434192 <== osd.7 v2:172.21.15.111:6816/3525126129 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b3c064030 con 0x7f9b200ad690 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] conn(0x7f9b200ad690 msgr2=0x7f9b200afb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] conn(0x7f9b200ad690 0x7f9b200afb30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9b200a3cd0 msgr2=0x7f9b200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9b200a3cd0 0x7f9b200a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c109c30 tx=0x7f9b24005bf0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 msgr2=0x7f9b3c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 --2- 172.21.15.97:0/3844434192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b3c1035b0 0x7f9b3c19f600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c970 tx=0x7f9b2c00cd30 comp rx=0 tx=0).stop 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41000700 1 -- 172.21.15.97:0/3844434192 reap_dead start 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 shutdown_connections 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 >> 172.21.15.97:0/3844434192 conn(0x7f9b3c0feea0 msgr2=0x7f9b3c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 shutdown_connections 2024-04-26T21:31:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.044+0000 7f9b41a00700 1 -- 172.21.15.97:0/3844434192 wait complete. 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 -- 172.21.15.97:0/4213946154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075f70 msgr2=0x7efd901140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 --2- 172.21.15.97:0/4213946154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075f70 0x7efd901140b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efd78009990 tx=0x7efd78009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 -- 172.21.15.97:0/4213946154 shutdown_connections 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 --2- 172.21.15.97:0/4213946154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd901145f0 0x7efd901169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 --2- 172.21.15.97:0/4213946154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd90075650 0x7efd90075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 --2- 172.21.15.97:0/4213946154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075f70 0x7efd901140b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 -- 172.21.15.97:0/4213946154 >> 172.21.15.97:0/4213946154 conn(0x7efd9007a580 msgr2=0x7efd9007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 -- 172.21.15.97:0/4213946154 shutdown_connections 2024-04-26T21:31:33.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.931+0000 7efd94a00700 1 -- 172.21.15.97:0/4213946154 wait complete. 2024-04-26T21:31:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 Processor -- start 2024-04-26T21:31:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 -- start start 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 0x7efd901a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd90075f70 0x7efd901a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd901145f0 0x7efd901a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd90119af0 con 0x7efd90075650 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd90119970 con 0x7efd901145f0 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.933+0000 7efd94a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd90119c70 con 0x7efd90075f70 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 0x7efd901a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 0x7efd901a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48880/0 (socket says 172.21.15.97:48880) 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 -- 172.21.15.97:0/726766963 learned_addr learned my addr 172.21.15.97:0/726766963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:33.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd85400700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd90075f70 0x7efd901a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8e800700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd901145f0 0x7efd901a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 -- 172.21.15.97:0/726766963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd90075f70 msgr2=0x7efd901a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd90075f70 0x7efd901a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 -- 172.21.15.97:0/726766963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd901145f0 msgr2=0x7efd901a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd901145f0 0x7efd901a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.934+0000 7efd8de00700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd901a50f0 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd8de00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 0x7efd901a00e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd80007f10 tx=0x7efd8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000dbd0 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd8000dd70 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80005950 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901a53e0 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.935+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901acc60 con 0x7efd90075650 2024-04-26T21:31:33.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.936+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd90110af0 con 0x7efd90075650 2024-04-26T21:31:33.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.938+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efd8001a020 con 0x7efd90075650 2024-04-26T21:31:33.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd87400700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efd740a3d50 0x7efd740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7efd800d03b0 con 0x7efd90075650 2024-04-26T21:31:33.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd85400700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efd740a3d50 0x7efd740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd87400700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] conn(0x7efd740ad710 0x7efd740afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:33.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd80021e70 con 0x7efd740ad710 2024-04-26T21:31:33.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.940+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efd800d0760 con 0x7efd90075650 2024-04-26T21:31:33.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.941+0000 7efd8e800700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] conn(0x7efd740ad710 0x7efd740afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:33.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.941+0000 7efd85400700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efd740a3d50 0x7efd740a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd78003760 tx=0x7efd78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.941+0000 7efd8e800700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] conn(0x7efd740ad710 0x7efd740afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:33.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.942+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== osd.3 v2:172.21.15.97:6810/92668618 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd80021e70 con 0x7efd740ad710 2024-04-26T21:31:33.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.960+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 --> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd90076d40 con 0x7efd740ad710 2024-04-26T21:31:33.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.960+0000 7efd87400700 1 -- 172.21.15.97:0/726766963 <== osd.3 v2:172.21.15.97:6810/92668618 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd90076d40 con 0x7efd740ad710 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] conn(0x7efd740ad710 msgr2=0x7efd740afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] conn(0x7efd740ad710 0x7efd740afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efd740a3d50 msgr2=0x7efd740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efd740a3d50 0x7efd740a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd78003760 tx=0x7efd78005fd0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 msgr2=0x7efd901a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 --2- 172.21.15.97:0/726766963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd90075650 0x7efd901a00e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd80007f10 tx=0x7efd8000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd8e800700 1 -- 172.21.15.97:0/726766963 reap_dead start 2024-04-26T21:31:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 shutdown_connections 2024-04-26T21:31:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 >> 172.21.15.97:0/726766963 conn(0x7efd9007a580 msgr2=0x7efd90110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 shutdown_connections 2024-04-26T21:31:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.961+0000 7efd94a00700 1 -- 172.21.15.97:0/726766963 wait complete. 2024-04-26T21:31:33.978 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:33 smithi097 ceph-mon[28127]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:31:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:33 smithi143 ceph-mon[31461]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:31:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:33 smithi111 ceph-mon[31001]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-26T21:31:34.408 INFO:teuthology.orchestra.run.smithi097.stdout:227633266740 2024-04-26T21:31:34.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T21:31:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 -- 172.21.15.97:0/607481719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 msgr2=0x7f2e78109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/607481719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e78109870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009990 tx=0x7f2e68009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 -- 172.21.15.97:0/607481719 shutdown_connections 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/607481719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e7806c360 0x7f2e781102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/607481719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e78109870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/607481719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e7806b9c0 0x7f2e7806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 -- 172.21.15.97:0/607481719 >> 172.21.15.97:0/607481719 conn(0x7f2e78079290 msgr2=0x7f2e780796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 -- 172.21.15.97:0/607481719 shutdown_connections 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.719+0000 7f2e7ee00700 1 -- 172.21.15.97:0/607481719 wait complete. 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.720+0000 7f2e7ee00700 1 Processor -- start 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.720+0000 7f2e7ee00700 1 -- start start 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.720+0000 7f2e7ee00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e7806b9c0 0x7f2e781a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ee00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e7806c360 0x7f2e781a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ee00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e7819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ee00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113500 con 0x7f2e78109490 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ee00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113380 con 0x7f2e7806b9c0 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ee00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113680 con 0x7f2e7806c360 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e7819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e7819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48868/0 (socket says 172.21.15.97:48868) 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 -- 172.21.15.97:0/1457084426 learned_addr learned my addr 172.21.15.97:0/1457084426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e77e00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e7806b9c0 0x7f2e781a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e77400700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e7806c360 0x7f2e781a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 -- 172.21.15.97:0/1457084426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e7806c360 msgr2=0x7f2e781a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e7806c360 0x7f2e781a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 -- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e7806b9c0 msgr2=0x7f2e781a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e7806b9c0 0x7f2e781a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.721+0000 7f2e7ce00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7819c250 con 0x7f2e78109490 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e7ce00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e7819baf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000b970 tx=0x7f2e7000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:34.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70019070 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e7000ce40 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7819c540 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007370 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.722+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781a8800 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.723+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e78063b30 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.725+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2e7001c070 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.726+0000 7f2e74c00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e5c0a3cd0 0x7f2e5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f2e700d04d0 con 0x7f2e78109490 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e77e00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e5c0a3cd0 0x7f2e5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e74c00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] conn(0x7f2e5c0ad690 0x7f2e5c0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:34.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e70024e50 con 0x7f2e5c0ad690 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2e700d0880 con 0x7f2e78109490 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e77e00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e5c0a3cd0 0x7f2e5c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800b5c0 tx=0x7f2e6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.727+0000 7f2e77400700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] conn(0x7f2e5c0ad690 0x7f2e5c0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.728+0000 7f2e77400700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] conn(0x7f2e5c0ad690 0x7f2e5c0afb30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.731+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== osd.4 v2:172.21.15.111:6808/3555238271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e70024e50 con 0x7f2e5c0ad690 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.750+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 --> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e78063f30 con 0x7f2e5c0ad690 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.751+0000 7f2e74c00700 1 -- 172.21.15.97:0/1457084426 <== osd.4 v2:172.21.15.111:6808/3555238271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2e78063f30 con 0x7f2e5c0ad690 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.751+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] conn(0x7f2e5c0ad690 msgr2=0x7f2e5c0afb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.751+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] conn(0x7f2e5c0ad690 0x7f2e5c0afb30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e5c0a3cd0 msgr2=0x7f2e5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f2e5c0a3cd0 0x7f2e5c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800b5c0 tx=0x7f2e6801a040 comp rx=0 tx=0).stop 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 msgr2=0x7f2e7819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:34.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 --2- 172.21.15.97:0/1457084426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e78109490 0x7f2e7819baf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000b970 tx=0x7f2e7000bd30 comp rx=0 tx=0).stop 2024-04-26T21:31:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ce00700 1 -- 172.21.15.97:0/1457084426 reap_dead start 2024-04-26T21:31:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 shutdown_connections 2024-04-26T21:31:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 >> 172.21.15.97:0/1457084426 conn(0x7f2e78079290 msgr2=0x7f2e7810e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 shutdown_connections 2024-04-26T21:31:34.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:33.752+0000 7f2e7ee00700 1 -- 172.21.15.97:0/1457084426 wait complete. 2024-04-26T21:31:34.845 INFO:teuthology.orchestra.run.smithi097.stdout:34359738466 2024-04-26T21:31:34.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T21:31:35.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:35 smithi097 ceph-mon[28127]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:31:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:35 smithi143 ceph-mon[31461]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:31:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:35 smithi111 ceph-mon[31001]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 -- 172.21.15.97:0/2202484286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 msgr2=0x7fd81c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 --2- 172.21.15.97:0/2202484286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c110320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd81400b0a0 tx=0x7fd81400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 -- 172.21.15.97:0/2202484286 shutdown_connections 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 --2- 172.21.15.97:0/2202484286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c110320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 --2- 172.21.15.97:0/2202484286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd81c1094e0 0x7fd81c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 --2- 172.21.15.97:0/2202484286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c1034a0 0x7fd81c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 -- 172.21.15.97:0/2202484286 >> 172.21.15.97:0/2202484286 conn(0x7fd81c07a5d0 msgr2=0x7fd81c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 -- 172.21.15.97:0/2202484286 shutdown_connections 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.253+0000 7fd822e00700 1 -- 172.21.15.97:0/2202484286 wait complete. 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 Processor -- start 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 -- start start 2024-04-26T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c1034a0 0x7fd81c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd81c1094e0 0x7fd81c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113500 con 0x7fd81c103e40 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113380 con 0x7fd81c1034a0 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.255+0000 7fd822e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113680 con 0x7fd81c1094e0 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48906/0 (socket says 172.21.15.97:48906) 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 -- 172.21.15.97:0/469113253 learned_addr learned my addr 172.21.15.97:0/469113253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd81be00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c1034a0 0x7fd81c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd820e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd81c1094e0 0x7fd81c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 -- 172.21.15.97:0/469113253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd81c1094e0 msgr2=0x7fd81c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd81c1094e0 0x7fd81c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 -- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c1034a0 msgr2=0x7fd81c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd81c1034a0 0x7fd81c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c19a020 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.256+0000 7fd813e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c19fbb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c9b0 tx=0x7fd80400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.257+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804007b00 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.257+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd804007ca0 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.257+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804018660 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.257+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c19a2b0 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.257+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a64d0 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.258+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd81c063b70 con 0x7fd81c103e40 2024-04-26T21:31:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.260+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd80400f450 con 0x7fd81c103e40 2024-04-26T21:31:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.262+0000 7fd819600700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd8000a3cd0 0x7fd8000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.263+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fd8040d0970 con 0x7fd81c103e40 2024-04-26T21:31:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.263+0000 7fd81be00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd8000a3cd0 0x7fd8000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.263+0000 7fd819600700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] conn(0x7fd8000ad690 0x7fd8000afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.263+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd80401fe50 con 0x7fd8000ad690 2024-04-26T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.263+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd8040d0e50 con 0x7fd81c103e40 2024-04-26T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.264+0000 7fd81be00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd8000a3cd0 0x7fd8000a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005fd0 tx=0x7fd80c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.264+0000 7fd820e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] conn(0x7fd8000ad690 0x7fd8000afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.265+0000 7fd820e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] conn(0x7fd8000ad690 0x7fd8000afb30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.267+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== osd.1 v2:172.21.15.111:6800/4100839926 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd80401fe50 con 0x7fd8000ad690 2024-04-26T21:31:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.286+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 --> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd81c063ff0 con 0x7fd8000ad690 2024-04-26T21:31:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.287+0000 7fd819600700 1 -- 172.21.15.97:0/469113253 <== osd.1 v2:172.21.15.111:6800/4100839926 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd81c063ff0 con 0x7fd8000ad690 2024-04-26T21:31:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] conn(0x7fd8000ad690 msgr2=0x7fd8000afb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] conn(0x7fd8000ad690 0x7fd8000afb30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd8000a3cd0 msgr2=0x7fd8000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fd8000a3cd0 0x7fd8000a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005fd0 tx=0x7fd80c005d90 comp rx=0 tx=0).stop 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 msgr2=0x7fd81c19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 --2- 172.21.15.97:0/469113253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd81c103e40 0x7fd81c19fbb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c9b0 tx=0x7fd80400cd70 comp rx=0 tx=0).stop 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd820e00700 1 -- 172.21.15.97:0/469113253 reap_dead start 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 shutdown_connections 2024-04-26T21:31:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 >> 172.21.15.97:0/469113253 conn(0x7fd81c07a5d0 msgr2=0x7fd81c101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.288+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 shutdown_connections 2024-04-26T21:31:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.289+0000 7fd822e00700 1 -- 172.21.15.97:0/469113253 wait complete. 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.356+0000 7f3e75600700 1 -- 172.21.15.97:0/3685196580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075650 msgr2=0x7f3e70075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.356+0000 7f3e75600700 1 --2- 172.21.15.97:0/3685196580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075650 0x7f3e70075a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000b0a0 tx=0x7f3e6000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 -- 172.21.15.97:0/3685196580 shutdown_connections 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 --2- 172.21.15.97:0/3685196580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7010fe20 0x7f3e70112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 --2- 172.21.15.97:0/3685196580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075650 0x7f3e70075a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 --2- 172.21.15.97:0/3685196580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e70075f70 0x7f3e7010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 -- 172.21.15.97:0/3685196580 >> 172.21.15.97:0/3685196580 conn(0x7f3e7007a580 msgr2=0x7f3e7007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 -- 172.21.15.97:0/3685196580 shutdown_connections 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.357+0000 7f3e75600700 1 -- 172.21.15.97:0/3685196580 wait complete. 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 Processor -- start 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 -- start start 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e70075650 0x7f3e7019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 0x7f3e7019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e7010fe20 0x7f3e701a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701157f0 con 0x7f3e70075f70 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70115670 con 0x7f3e7010fe20 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e75600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70115970 con 0x7f3e70075650 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.358+0000 7f3e66000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 0x7f3e7019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 0x7f3e7019c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48924/0 (socket says 172.21.15.97:48924) 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 -- 172.21.15.97:0/1822901608 learned_addr learned my addr 172.21.15.97:0/1822901608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e6ea00700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e70075650 0x7f3e7019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e6f400700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e7010fe20 0x7f3e701a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 -- 172.21.15.97:0/1822901608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e70075650 msgr2=0x7f3e7019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e70075650 0x7f3e7019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 -- 172.21.15.97:0/1822901608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e7010fe20 msgr2=0x7f3e701a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e7010fe20 0x7f3e701a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e701a0dc0 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.359+0000 7f3e66000700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 0x7f3e7019c2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b500 tx=0x7f3e5800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.360+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58015020 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.360+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e58011cc0 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.360+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e701a10b0 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.360+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58010ca0 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.360+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e701a8930 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.361+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e70063bb0 con 0x7f3e70075f70 2024-04-26T21:31:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.363+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e58018020 con 0x7f3e70075f70 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.364+0000 7f3e67e00700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e540a3cd0 0x7f3e540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3e580d0200 con 0x7f3e70075f70 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e6ea00700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e540a3cd0 0x7f3e540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e67e00700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] conn(0x7f3e540ad690 0x7f3e540afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e540b0200 con 0x7f3e540ad690 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3e580d06e0 con 0x7f3e70075f70 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.365+0000 7f3e6f400700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] conn(0x7f3e540ad690 0x7f3e540afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.366+0000 7f3e6ea00700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e540a3cd0 0x7f3e540a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009e60 tx=0x7f3e6000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.366+0000 7f3e6f400700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] conn(0x7f3e540ad690 0x7f3e540afb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.367+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== osd.6 v2:172.21.15.97:6818/3234116941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e540b0200 con 0x7f3e540ad690 2024-04-26T21:31:36.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.386+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 --> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e70063fb0 con 0x7f3e540ad690 2024-04-26T21:31:36.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.386+0000 7f3e67e00700 1 -- 172.21.15.97:0/1822901608 <== osd.6 v2:172.21.15.97:6818/3234116941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3e70063fb0 con 0x7f3e540ad690 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] conn(0x7f3e540ad690 msgr2=0x7f3e540afb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] conn(0x7f3e540ad690 0x7f3e540afb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e540a3cd0 msgr2=0x7f3e540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e540a3cd0 0x7f3e540a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009e60 tx=0x7f3e6000b7b0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 msgr2=0x7f3e7019c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 --2- 172.21.15.97:0/1822901608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e70075f70 0x7f3e7019c2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b500 tx=0x7f3e5800b8c0 comp rx=0 tx=0).stop 2024-04-26T21:31:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e6f400700 1 -- 172.21.15.97:0/1822901608 reap_dead start 2024-04-26T21:31:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 shutdown_connections 2024-04-26T21:31:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 >> 172.21.15.97:0/1822901608 conn(0x7f3e7007a580 msgr2=0x7f3e70100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 shutdown_connections 2024-04-26T21:31:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:36.387+0000 7f3e75600700 1 -- 172.21.15.97:0/1822901608 wait complete. 2024-04-26T21:31:37.979 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:37 smithi097 ceph-mon[28127]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-26T21:31:38.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:37 smithi143 ceph-mon[31461]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-26T21:31:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:37 smithi111 ceph-mon[31001]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 -- 172.21.15.97:0/3514686884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c10fe20 msgr2=0x7f507c112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 --2- 172.21.15.97:0/3514686884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c10fe20 0x7f507c112210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f507400b0a0 tx=0x7f507400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 -- 172.21.15.97:0/3514686884 shutdown_connections 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 --2- 172.21.15.97:0/3514686884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f507c075650 0x7f507c075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 --2- 172.21.15.97:0/3514686884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c10fe20 0x7f507c112210 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 --2- 172.21.15.97:0/3514686884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c075f70 0x7f507c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 -- 172.21.15.97:0/3514686884 >> 172.21.15.97:0/3514686884 conn(0x7f507c07a5d0 msgr2=0x7f507c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.082+0000 7f5081c00700 1 -- 172.21.15.97:0/3514686884 shutdown_connections 2024-04-26T21:31:39.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.083+0000 7f5081c00700 1 -- 172.21.15.97:0/3514686884 wait complete. 2024-04-26T21:31:39.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.084+0000 7f5081c00700 1 Processor -- start 2024-04-26T21:31:39.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.084+0000 7f5081c00700 1 -- start start 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f507c075650 0x7f507c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 0x7f507c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c10fe20 0x7f507c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f507c115760 con 0x7f507c075f70 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1155e0 con 0x7f507c075650 2024-04-26T21:31:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f5081c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1158e0 con 0x7f507c10fe20 2024-04-26T21:31:39.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f507a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 0x7f507c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f507a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 0x7f507c1a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48950/0 (socket says 172.21.15.97:48950) 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.085+0000 7f507a000700 1 -- 172.21.15.97:0/2957205434 learned_addr learned my addr 172.21.15.97:0/2957205434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507aa00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f507c075650 0x7f507c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507b400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c10fe20 0x7f507c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 -- 172.21.15.97:0/2957205434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c10fe20 msgr2=0x7f507c19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c10fe20 0x7f507c19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 -- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f507c075650 msgr2=0x7f507c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f507c075650 0x7f507c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c19c200 con 0x7f507c075f70 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507b400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f507c10fe20 0x7f507c19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.086+0000 7f507a000700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 0x7f507c1a1e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f506800d980 tx=0x7f506800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.087+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068017070 con 0x7f507c075f70 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.087+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5068005020 con 0x7f507c075f70 2024-04-26T21:31:39.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.087+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068009ab0 con 0x7f507c075f70 2024-04-26T21:31:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.087+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c19c4f0 con 0x7f507c075f70 2024-04-26T21:31:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.087+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a87b0 con 0x7f507c075f70 2024-04-26T21:31:39.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.088+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f507c063b70 con 0x7f507c075f70 2024-04-26T21:31:39.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.091+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50680051d0 con 0x7f507c075f70 2024-04-26T21:31:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.092+0000 7f5073400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f50600a3d50 0x7f50600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.093+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f50680d0900 con 0x7f507c075f70 2024-04-26T21:31:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.093+0000 7f507aa00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f50600a3d50 0x7f50600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.093+0000 7f5073400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] conn(0x7f50600ad710 0x7f50600afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.093+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50600b0280 con 0x7f50600ad710 2024-04-26T21:31:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.093+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f50680d0cb0 con 0x7f507c075f70 2024-04-26T21:31:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.094+0000 7f507b400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] conn(0x7f50600ad710 0x7f50600afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:39.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.094+0000 7f507aa00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f50600a3d50 0x7f50600a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f507c19cbe0 tx=0x7f506c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:39.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.095+0000 7f507b400700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] conn(0x7f50600ad710 0x7f50600afbb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.098+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== osd.10 v2:172.21.15.111:6824/3431248583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50600b0280 con 0x7f50600ad710 2024-04-26T21:31:39.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.115+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 --> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f507c063ff0 con 0x7f50600ad710 2024-04-26T21:31:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.116+0000 7f5073400700 1 -- 172.21.15.97:0/2957205434 <== osd.10 v2:172.21.15.111:6824/3431248583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f507c063ff0 con 0x7f50600ad710 2024-04-26T21:31:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] conn(0x7f50600ad710 msgr2=0x7f50600afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] conn(0x7f50600ad710 0x7f50600afbb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f50600a3d50 msgr2=0x7f50600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f50600a3d50 0x7f50600a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f507c19cbe0 tx=0x7f506c00b410 comp rx=0 tx=0).stop 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 msgr2=0x7f507c1a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 --2- 172.21.15.97:0/2957205434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f507c075f70 0x7f507c1a1e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f506800d980 tx=0x7f506800dc90 comp rx=0 tx=0).stop 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f507b400700 1 -- 172.21.15.97:0/2957205434 reap_dead start 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 shutdown_connections 2024-04-26T21:31:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 >> 172.21.15.97:0/2957205434 conn(0x7f507c07a5d0 msgr2=0x7f507c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 shutdown_connections 2024-04-26T21:31:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:39.117+0000 7f5081c00700 1 -- 172.21.15.97:0/2957205434 wait complete. 2024-04-26T21:31:39.495 INFO:teuthology.orchestra.run.smithi097.stdout:55834574943 2024-04-26T21:31:39.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T21:31:40.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:39 smithi143 ceph-mon[31461]: pgmap v389: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:40.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:39 smithi097 ceph-mon[28127]: pgmap v389: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:39 smithi111 ceph-mon[31001]: pgmap v389: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:41.361 INFO:teuthology.orchestra.run.smithi097.stdout:206158430267 2024-04-26T21:31:41.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T21:31:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:41 smithi143 ceph-mon[31461]: pgmap v390: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:31:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:41 smithi097 ceph-mon[28127]: pgmap v390: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:31:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:41 smithi111 ceph-mon[31001]: pgmap v390: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:31:42.268 INFO:teuthology.orchestra.run.smithi097.stdout:111669149776 2024-04-26T21:31:42.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 -- 172.21.15.97:0/3784855289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c68103e30 msgr2=0x7f9c68110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 --2- 172.21.15.97:0/3784855289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c68103e30 0x7f9c68110290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000b0a0 tx=0x7f9c6000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 -- 172.21.15.97:0/3784855289 shutdown_connections 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 --2- 172.21.15.97:0/3784855289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c681094c0 0x7f9c681098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 --2- 172.21.15.97:0/3784855289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c68103e30 0x7f9c68110290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 --2- 172.21.15.97:0/3784855289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c68103470 0x7f9c681038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 -- 172.21.15.97:0/3784855289 >> 172.21.15.97:0/3784855289 conn(0x7f9c6807a580 msgr2=0x7f9c6807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 -- 172.21.15.97:0/3784855289 shutdown_connections 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.541+0000 7f9c70200700 1 -- 172.21.15.97:0/3784855289 wait complete. 2024-04-26T21:31:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 Processor -- start 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 -- start start 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c68103470 0x7f9c6819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c68103e30 0x7f9c6819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 0x7f9c681a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681135c0 con 0x7f9c681094c0 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113440 con 0x7f9c68103470 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.542+0000 7f9c70200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113740 con 0x7f9c68103e30 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 0x7f9c681a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 0x7f9c681a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48978/0 (socket says 172.21.15.97:48978) 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 -- 172.21.15.97:0/2864439945 learned_addr learned my addr 172.21.15.97:0/2864439945 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6ce00700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c68103e30 0x7f9c6819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 -- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c68103e30 msgr2=0x7f9c6819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c68103e30 0x7f9c6819c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6d800700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c68103470 0x7f9c6819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 -- 172.21.15.97:0/2864439945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c68103470 msgr2=0x7f9c6819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c68103470 0x7f9c6819bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c681a0da0 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.543+0000 7f9c6e200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 0x7f9c681a0660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c60004dc0 tx=0x7f9c60004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.544+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60021020 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.544+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c60007cf0 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.544+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c681a1030 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.544+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60005370 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.544+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c681a8910 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.545+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c68063bb0 con 0x7f9c681094c0 2024-04-26T21:31:42.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.547+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c60024020 con 0x7f9c681094c0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c66000700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c500a3d50 0x7f9c500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9c600d1e70 con 0x7f9c681094c0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c6d800700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c500a3d50 0x7f9c500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c66000700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] conn(0x7f9c500ad770 0x7f9c500afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c6002fe50 con 0x7f9c500ad770 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.548+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9c600d2220 con 0x7f9c681094c0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.549+0000 7f9c6d800700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c500a3d50 0x7f9c500a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c006fd0 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.565+0000 7f9c6ce00700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] conn(0x7f9c500ad770 0x7f9c500afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.566+0000 7f9c6ce00700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] conn(0x7f9c500ad770 0x7f9c500afc10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.569+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== osd.8 v2:172.21.15.143:6816/2919334001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c6002fe50 con 0x7f9c500ad770 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.587+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 --> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c68064030 con 0x7f9c500ad770 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.588+0000 7f9c66000700 1 -- 172.21.15.97:0/2864439945 <== osd.8 v2:172.21.15.143:6816/2919334001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c68064030 con 0x7f9c500ad770 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.588+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] conn(0x7f9c500ad770 msgr2=0x7f9c500afc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.588+0000 7f9c70200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] conn(0x7f9c500ad770 0x7f9c500afc10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c500a3d50 msgr2=0x7f9c500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c500a3d50 0x7f9c500a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c006fd0 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 msgr2=0x7f9c681a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 --2- 172.21.15.97:0/2864439945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c681094c0 0x7f9c681a0660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c60004dc0 tx=0x7f9c60004ea0 comp rx=0 tx=0).stop 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c6e200700 1 -- 172.21.15.97:0/2864439945 reap_dead start 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 shutdown_connections 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 >> 172.21.15.97:0/2864439945 conn(0x7f9c6807a580 msgr2=0x7f9c6810ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 shutdown_connections 2024-04-26T21:31:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:40.589+0000 7f9c70200700 1 -- 172.21.15.97:0/2864439945 wait complete. 2024-04-26T21:31:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 -- 172.21.15.97:0/2093489753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906410d950 msgr2=0x7f906410fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 --2- 172.21.15.97:0/2093489753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906410d950 0x7f906410fdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f905c00b0a0 tx=0x7f905c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 -- 172.21.15.97:0/2093489753 shutdown_connections 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 --2- 172.21.15.97:0/2093489753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9064104570 0x7f9064104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 --2- 172.21.15.97:0/2093489753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906410d950 0x7f906410fdf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 --2- 172.21.15.97:0/2093489753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9064104e90 0x7f906410d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 -- 172.21.15.97:0/2093489753 >> 172.21.15.97:0/2093489753 conn(0x7f90640fdd40 msgr2=0x7f9064100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 -- 172.21.15.97:0/2093489753 shutdown_connections 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.462+0000 7f9068c00700 1 -- 172.21.15.97:0/2093489753 wait complete. 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.464+0000 7f9068c00700 1 Processor -- start 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.464+0000 7f9068c00700 1 -- start start 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.464+0000 7f9068c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 0x7f90641a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9068c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9064104e90 0x7f90641a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9068c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906410d950 0x7f906419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9068c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9064113430 con 0x7f9064104570 2024-04-26T21:31:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9068c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90641132b0 con 0x7f9064104e90 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9068c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90641135b0 con 0x7f906410d950 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 0x7f90641a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 0x7f90641a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56756/0 (socket says 172.21.15.97:56756) 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 -- 172.21.15.97:0/3560941899 learned_addr learned my addr 172.21.15.97:0/3560941899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9062a00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9064104e90 0x7f90641a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 -- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906410d950 msgr2=0x7f906419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906410d950 0x7f906419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 -- 172.21.15.97:0/3560941899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9064104e90 msgr2=0x7f90641a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9064104e90 0x7f90641a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.465+0000 7f9063400700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906419c0b0 con 0x7f9064104570 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f9063400700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 0x7f90641a16f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f905400c9b0 tx=0x7f905400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054007b00 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9054007ca0 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906419c3a0 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054018660 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.466+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a85c0 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.467+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9064103cb0 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.469+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f905400f450 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f905be00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90480a3cd0 0x7f90480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f90540d0a70 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f9062a00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90480a3cd0 0x7f90480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f905be00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] conn(0x7f90480ad690 0x7f90480afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f905401fe50 con 0x7f90480ad690 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.471+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f90540d0e20 con 0x7f9064104570 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.472+0000 7f9062a00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90480a3cd0 0x7f90480a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9050005fd0 tx=0x7f9050005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.472+0000 7f9063e00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] conn(0x7f90480ad690 0x7f90480afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.473+0000 7f9063e00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] conn(0x7f90480ad690 0x7f90480afb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.475+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== osd.5 v2:172.21.15.143:6808/1354252557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f905401fe50 con 0x7f90480ad690 2024-04-26T21:31:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.492+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 --> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f906406b980 con 0x7f90480ad690 2024-04-26T21:31:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.493+0000 7f905be00700 1 -- 172.21.15.97:0/3560941899 <== osd.5 v2:172.21.15.143:6808/1354252557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f906406b980 con 0x7f90480ad690 2024-04-26T21:31:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] conn(0x7f90480ad690 msgr2=0x7f90480afb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] conn(0x7f90480ad690 0x7f90480afb30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90480a3cd0 msgr2=0x7f90480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90480a3cd0 0x7f90480a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9050005fd0 tx=0x7f9050005db0 comp rx=0 tx=0).stop 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 msgr2=0x7f90641a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9068c00700 1 --2- 172.21.15.97:0/3560941899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9064104570 0x7f90641a16f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f905400c9b0 tx=0x7f905400cd70 comp rx=0 tx=0).stop 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.494+0000 7f9063e00700 1 -- 172.21.15.97:0/3560941899 reap_dead start 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.495+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 shutdown_connections 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.495+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 >> 172.21.15.97:0/3560941899 conn(0x7f90640fdd40 msgr2=0x7f906410fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.495+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 shutdown_connections 2024-04-26T21:31:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:41.495+0000 7f9068c00700 1 -- 172.21.15.97:0/3560941899 wait complete. 2024-04-26T21:31:44.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:43 smithi143 ceph-mon[31461]: pgmap v391: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:31:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:43 smithi097 ceph-mon[28127]: pgmap v391: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:31:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:43 smithi111 ceph-mon[31001]: pgmap v391: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:31:44.179 INFO:teuthology.orchestra.run.smithi097.stdout:322122547232 2024-04-26T21:31:44.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T21:31:45.715 INFO:teuthology.orchestra.run.smithi097.stdout:171798691908 2024-04-26T21:31:45.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T21:31:45.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:45 smithi097 ceph-mon[28127]: pgmap v392: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 -- 172.21.15.97:0/1782434908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e0103470 msgr2=0x7fb0e01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 --2- 172.21.15.97:0/1782434908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e0103470 0x7fb0e01038f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009990 tx=0x7fb0cc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 -- 172.21.15.97:0/1782434908 shutdown_connections 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 --2- 172.21.15.97:0/1782434908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e01094f0 0x7fb0e01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 --2- 172.21.15.97:0/1782434908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e0103e30 0x7fb0e01102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 --2- 172.21.15.97:0/1782434908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e0103470 0x7fb0e01038f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 -- 172.21.15.97:0/1782434908 >> 172.21.15.97:0/1782434908 conn(0x7fb0e007a5d0 msgr2=0x7fb0e007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 -- 172.21.15.97:0/1782434908 shutdown_connections 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.740+0000 7fb0e5400700 1 -- 172.21.15.97:0/1782434908 wait complete. 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.741+0000 7fb0e5400700 1 Processor -- start 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.741+0000 7fb0e5400700 1 -- start start 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.741+0000 7fb0e5400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0103470 0x7fb0e0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0e5400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e0103e30 0x7fb0e0075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0e5400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 0x7fb0e00761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0e5400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01134f0 con 0x7fb0e01094f0 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0e5400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0113370 con 0x7fb0e0103470 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0e5400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0113670 con 0x7fb0e0103e30 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 0x7fb0e00761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 0x7fb0e00761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56760/0 (socket says 172.21.15.97:56760) 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 -- 172.21.15.97:0/3568639267 learned_addr learned my addr 172.21.15.97:0/3568639267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0de800700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0103470 0x7fb0e0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0dde00700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e0103e30 0x7fb0e0075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 -- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e0103e30 msgr2=0x7fb0e0075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e0103e30 0x7fb0e0075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 -- 172.21.15.97:0/3568639267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0103470 msgr2=0x7fb0e0075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0103470 0x7fb0e0075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0df200700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e01a8100 con 0x7fb0e01094f0 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.742+0000 7fb0de800700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0e0103470 0x7fb0e0075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0df200700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 0x7fb0e00761a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800ab50 tx=0x7fb0d800af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8017020 con 0x7fb0e01094f0 2024-04-26T21:31:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0d8004bb0 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8005280 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e01a83f0 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.743+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e01a88e0 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.744+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0e0063b30 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.746+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0d800c6d0 con 0x7fb0e01094f0 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0d7400700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb0c40a3d50 0x7fb0c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fb0d80cfc10 con 0x7fb0e01094f0 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0de800700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb0c40a3d50 0x7fb0c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0d7400700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] conn(0x7fb0c40ad710 0x7fb0c40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0d8008e50 con 0x7fb0c40ad710 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.748+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb0d80d42a0 con 0x7fb0e01094f0 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.749+0000 7fb0de800700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb0c40a3d50 0x7fb0c40a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0005fd0 tx=0x7fb0d000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.762+0000 7fb0dde00700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] conn(0x7fb0c40ad710 0x7fb0c40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.763+0000 7fb0dde00700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] conn(0x7fb0c40ad710 0x7fb0c40afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.766+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== osd.11 v2:172.21.15.143:6824/921154932 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0d8008e50 con 0x7fb0c40ad710 2024-04-26T21:31:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.786+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 --> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb0e0063f30 con 0x7fb0c40ad710 2024-04-26T21:31:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0d7400700 1 -- 172.21.15.97:0/3568639267 <== osd.11 v2:172.21.15.143:6824/921154932 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb0e0063f30 con 0x7fb0c40ad710 2024-04-26T21:31:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] conn(0x7fb0c40ad710 msgr2=0x7fb0c40afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] conn(0x7fb0c40ad710 0x7fb0c40afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb0c40a3d50 msgr2=0x7fb0c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fb0c40a3d50 0x7fb0c40a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0005fd0 tx=0x7fb0d000a5e0 comp rx=0 tx=0).stop 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 msgr2=0x7fb0e00761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0e5400700 1 --2- 172.21.15.97:0/3568639267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e01094f0 0x7fb0e00761a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800ab50 tx=0x7fb0d800af10 comp rx=0 tx=0).stop 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.787+0000 7fb0df200700 1 -- 172.21.15.97:0/3568639267 reap_dead start 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.788+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 shutdown_connections 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.788+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 >> 172.21.15.97:0/3568639267 conn(0x7fb0e007a5d0 msgr2=0x7fb0e01022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.788+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 shutdown_connections 2024-04-26T21:31:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:45.788+0000 7fb0e5400700 1 -- 172.21.15.97:0/3568639267 wait complete. 2024-04-26T21:31:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:45 smithi143 ceph-mon[31461]: pgmap v392: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:45 smithi111 ceph-mon[31001]: pgmap v392: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:46.239 INFO:teuthology.orchestra.run.smithi097.stdout:266287972399 2024-04-26T21:31:46.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T21:31:46.690 INFO:teuthology.orchestra.run.smithi097.stdout:146028888136 2024-04-26T21:31:46.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 -- 172.21.15.97:0/1671105383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1148109a50 msgr2=0x7f1148109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 --2- 172.21.15.97:0/1671105383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1148109a50 0x7f1148109ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1134009990 tx=0x7f1134009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 -- 172.21.15.97:0/1671105383 shutdown_connections 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 --2- 172.21.15.97:0/1671105383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f114810fa50 0x7f114810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 --2- 172.21.15.97:0/1671105383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114810a410 0x7f1148116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 --2- 172.21.15.97:0/1671105383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1148109a50 0x7f1148109ed0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 -- 172.21.15.97:0/1671105383 >> 172.21.15.97:0/1671105383 conn(0x7f1148076bf0 msgr2=0x7f1148108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 -- 172.21.15.97:0/1671105383 shutdown_connections 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.582+0000 7f114da00700 1 -- 172.21.15.97:0/1671105383 wait complete. 2024-04-26T21:31:47.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.583+0000 7f114da00700 1 Processor -- start 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.583+0000 7f114da00700 1 -- start start 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1148109a50 0x7f114819ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 0x7f11481a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114810fa50 0x7f11481a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1148119b40 con 0x7f114810a410 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f11481199c0 con 0x7f1148109a50 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f114da00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1148119cc0 con 0x7f114810fa50 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 0x7f11481a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 0x7f11481a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56782/0 (socket says 172.21.15.97:56782) 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 -- 172.21.15.97:0/3146225389 learned_addr learned my addr 172.21.15.97:0/3146225389 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146a00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1148109a50 0x7f114819ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 -- 172.21.15.97:0/3146225389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114810fa50 msgr2=0x7f11481a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114810fa50 0x7f11481a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 -- 172.21.15.97:0/3146225389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1148109a50 msgr2=0x7f114819ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.584+0000 7f1146000700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1148109a50 0x7f114819ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f1146000700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11481a4fe0 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f1146000700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 0x7f11481a0510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1134000c00 tx=0x7f11340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134027070 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f113400fdb0 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340173f0 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11481a5270 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.585+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11481acb50 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.586+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1148063bb0 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.588+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f113402a070 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.589+0000 7f113f400700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f112c0a3d50 0x7f112c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f11340d1d10 con 0x7f114810a410 2024-04-26T21:31:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f1146a00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f112c0a3d50 0x7f112c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f113f400700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] conn(0x7f112c0ad710 0x7f112c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1134001d30 con 0x7f112c0ad710 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f11340d20c0 con 0x7f114810a410 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.590+0000 7f1147400700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] conn(0x7f112c0ad710 0x7f112c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.591+0000 7f1146a00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f112c0a3d50 0x7f112c0a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1138005ea0 tx=0x7f1138005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.591+0000 7f1147400700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] conn(0x7f112c0ad710 0x7f112c0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.592+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== osd.9 v2:172.21.15.97:6826/3567005259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1134001d30 con 0x7f112c0ad710 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.611+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 --> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1148063fb0 con 0x7f112c0ad710 2024-04-26T21:31:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.611+0000 7f113f400700 1 -- 172.21.15.97:0/3146225389 <== osd.9 v2:172.21.15.97:6826/3567005259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1148063fb0 con 0x7f112c0ad710 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] conn(0x7f112c0ad710 msgr2=0x7f112c0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] conn(0x7f112c0ad710 0x7f112c0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f112c0a3d50 msgr2=0x7f112c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f112c0a3d50 0x7f112c0a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1138005ea0 tx=0x7f1138005e30 comp rx=0 tx=0).stop 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 msgr2=0x7f11481a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 --2- 172.21.15.97:0/3146225389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f114810a410 0x7f11481a0510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1134000c00 tx=0x7f11340047b0 comp rx=0 tx=0).stop 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f1147400700 1 -- 172.21.15.97:0/3146225389 reap_dead start 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 shutdown_connections 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 >> 172.21.15.97:0/3146225389 conn(0x7f1148076bf0 msgr2=0x7f11480771d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 shutdown_connections 2024-04-26T21:31:47.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:47.612+0000 7f114da00700 1 -- 172.21.15.97:0/3146225389 wait complete. 2024-04-26T21:31:48.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:47 smithi143 ceph-mon[31461]: pgmap v393: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T21:31:48.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:47 smithi097 ceph-mon[28127]: pgmap v393: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T21:31:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:47 smithi111 ceph-mon[31001]: pgmap v393: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T21:31:48.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 -- 172.21.15.97:0/3478927535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 msgr2=0x7f3bf006b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/3478927535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf006b130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009990 tx=0x7f3be0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 -- 172.21.15.97:0/3478927535 shutdown_connections 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/3478927535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf0105e60 0x7f3bf01062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/3478927535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf006b130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/3478927535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bf006b670 0x7f3bf006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 -- 172.21.15.97:0/3478927535 >> 172.21.15.97:0/3478927535 conn(0x7f3bf00792d0 msgr2=0x7f3bf00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 -- 172.21.15.97:0/3478927535 shutdown_connections 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.910+0000 7f3bf6c00700 1 -- 172.21.15.97:0/3478927535 wait complete. 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.911+0000 7f3bf6c00700 1 Processor -- start 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 -- start start 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf006b670 0x7f3bf01a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bf0105e60 0x7f3bf01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf01a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:48.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0119a60 con 0x7f3bf0108350 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01198e0 con 0x7f3bf006b670 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0119be0 con 0x7f3bf0105e60 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf4c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf01a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf4c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf01a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56806/0 (socket says 172.21.15.97:56806) 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3bf4c00700 1 -- 172.21.15.97:0/4100800133 learned_addr learned my addr 172.21.15.97:0/4100800133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.912+0000 7f3befe00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf006b670 0x7f3bf01a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bef400700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bf0105e60 0x7f3bf01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:48.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 -- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bf0105e60 msgr2=0x7f3bf01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bf0105e60 0x7f3bf01a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 -- 172.21.15.97:0/4100800133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf006b670 msgr2=0x7f3bf01a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bf006b670 0x7f3bf01a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01a5060 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf4c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf01a4920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3be8007ee0 tx=0x7f3be800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be800dbd0 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3be800dd70 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be80058d0 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf01a5350 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.913+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01acbd0 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.914+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bf0101d70 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.916+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3be801a020 con 0x7f3bf0108350 2024-04-26T21:31:48.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3becc00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3bd40a3d50 0x7f3bd40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3be80d03b0 con 0x7f3bf0108350 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3befe00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3bd40a3d50 0x7f3bd40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3becc00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] conn(0x7f3bd40ad710 0x7f3bd40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bd40b0280 con 0x7f3bd40ad710 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.918+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3be80d0760 con 0x7f3bf0108350 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.919+0000 7f3befe00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3bd40a3d50 0x7f3bd40a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3be000b5c0 tx=0x7f3be001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.919+0000 7f3bef400700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] conn(0x7f3bd40ad710 0x7f3bd40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.920+0000 7f3bef400700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] conn(0x7f3bd40ad710 0x7f3bd40afbb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.923+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== osd.2 v2:172.21.15.143:6800/1100983067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bd40b0280 con 0x7f3bd40ad710 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.941+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 --> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bf0102940 con 0x7f3bd40ad710 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.942+0000 7f3becc00700 1 -- 172.21.15.97:0/4100800133 <== osd.2 v2:172.21.15.143:6800/1100983067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3bf0102940 con 0x7f3bd40ad710 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.942+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] conn(0x7f3bd40ad710 msgr2=0x7f3bd40afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.942+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] conn(0x7f3bd40ad710 0x7f3bd40afbb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3bd40a3d50 msgr2=0x7f3bd40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3bd40a3d50 0x7f3bd40a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3be000b5c0 tx=0x7f3be001a040 comp rx=0 tx=0).stop 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 msgr2=0x7f3bf01a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 --2- 172.21.15.97:0/4100800133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bf0108350 0x7f3bf01a4920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3be8007ee0 tx=0x7f3be800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf4c00700 1 -- 172.21.15.97:0/4100800133 reap_dead start 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 shutdown_connections 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 >> 172.21.15.97:0/4100800133 conn(0x7f3bf00792d0 msgr2=0x7f3bf0101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 shutdown_connections 2024-04-26T21:31:48.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:48.943+0000 7f3bf6c00700 1 -- 172.21.15.97:0/4100800133 wait complete. 2024-04-26T21:31:49.205 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:49.243 INFO:teuthology.orchestra.run.smithi097.stdout:356482285595 2024-04-26T21:31:49.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T21:31:49.737 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:49.741 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:49.746 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:49.747 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:49.788 INFO:teuthology.orchestra.run.smithi097.stdout:300647710761 2024-04-26T21:31:49.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T21:31:49.849 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:49 smithi097 ceph-mon[28127]: pgmap v394: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:31:50.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:49 smithi143 ceph-mon[31461]: pgmap v394: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:31:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:49 smithi111 ceph-mon[31001]: pgmap v394: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:31:50.254 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:50.255 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:50.261 INFO:teuthology.orchestra.run.smithi097.stdout:77309411419 2024-04-26T21:31:50.262 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:50.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T21:31:51.945 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:51 smithi097 ceph-mon[28127]: pgmap v395: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:52.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:51 smithi143 ceph-mon[31461]: pgmap v395: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:51 smithi111 ceph-mon[31001]: pgmap v395: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:54.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:53 smithi097 ceph-mon[28127]: pgmap v396: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:31:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:53 smithi111 ceph-mon[31001]: pgmap v396: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:31:54.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:53 smithi143 ceph-mon[31461]: pgmap v396: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:31:54.857 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:55.935 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:55 smithi097 ceph-mon[28127]: pgmap v397: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 -- 172.21.15.97:0/427099869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103e30 msgr2=0x7f7274110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 --2- 172.21.15.97:0/427099869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103e30 0x7f7274110290 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b0a0 tx=0x7f726c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 -- 172.21.15.97:0/427099869 shutdown_connections 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 --2- 172.21.15.97:0/427099869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103e30 0x7f7274110290 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 --2- 172.21.15.97:0/427099869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72741094c0 0x7f72741098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 --2- 172.21.15.97:0/427099869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7274103470 0x7f72741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 -- 172.21.15.97:0/427099869 >> 172.21.15.97:0/427099869 conn(0x7f727407a580 msgr2=0x7f727407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 -- 172.21.15.97:0/427099869 shutdown_connections 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.933+0000 7f7278c00700 1 -- 172.21.15.97:0/427099869 wait complete. 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 Processor -- start 2024-04-26T21:31:55.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 -- start start 2024-04-26T21:31:55.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7274103e30 0x7f72741a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 0x7f727419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72741134e0 con 0x7f7274103470 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7274113360 con 0x7f72741094c0 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.935+0000 7f7278c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7274113660 con 0x7f7274103e30 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7274103e30 0x7f72741a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60116/0 (socket says 172.21.15.97:60116) 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 -- 172.21.15.97:0/2796737318 learned_addr learned my addr 172.21.15.97:0/2796737318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7272800700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 0x7f727419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 -- 172.21.15.97:0/2796737318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7274103e30 msgr2=0x7f72741a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7274103e30 0x7f72741a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 -- 172.21.15.97:0/2796737318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 msgr2=0x7f727419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 0x7f727419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7271e00700 1 -- 172.21.15.97:0/2796737318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727419c1c0 con 0x7f7274103470 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.936+0000 7f7272800700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 0x7f727419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f7271e00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9c0 tx=0x7f726400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400cbd0 con 0x7f7274103470 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f726400ee60 con 0x7f7274103470 2024-04-26T21:31:55.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640054b0 con 0x7f7274103470 2024-04-26T21:31:55.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727419c450 con 0x7f7274103470 2024-04-26T21:31:55.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.937+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741a8770 con 0x7f7274103470 2024-04-26T21:31:55.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.939+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727406a110 con 0x7f7274103470 2024-04-26T21:31:55.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.941+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7264004750 con 0x7f7274103470 2024-04-26T21:31:55.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.942+0000 7f726aa00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 0x7f72580a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:31:55.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.942+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f72640d06d0 con 0x7f7274103470 2024-04-26T21:31:55.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.942+0000 7f7271400700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 0x7f72580a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:31:55.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.943+0000 7f7271400700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 0x7f72580a6310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727407aec0 tx=0x7f72600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:31:55.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:55.947+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f726409cf60 con 0x7f7274103470 2024-04-26T21:31:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:55 smithi111 ceph-mon[31001]: pgmap v397: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:31:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.190+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f727402e490 con 0x7f7274103470 2024-04-26T21:31:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.190+0000 7f726aa00700 1 -- 172.21.15.97:0/2796737318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f726409c860 con 0x7f7274103470 2024-04-26T21:31:56.192 INFO:teuthology.orchestra.run.smithi097.stdout:34359738472 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.193+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 msgr2=0x7f72580a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 0x7f72580a6310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727407aec0 tx=0x7f72600073d0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 msgr2=0x7f72741a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9c0 tx=0x7f726400ecd0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 shutdown_connections 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f72580a3e50 0x7f72580a6310 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72741094c0 0x7f727419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7274103470 0x7f72741a18a0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9c0 tx=0x7f726400ecd0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 --2- 172.21.15.97:0/2796737318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7274103e30 0x7f72741a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:31:56.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 >> 172.21.15.97:0/2796737318 conn(0x7f727407a580 msgr2=0x7f727410e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:31:56.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 shutdown_connections 2024-04-26T21:31:56.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:31:56.194+0000 7f7278c00700 1 -- 172.21.15.97:0/2796737318 wait complete. 2024-04-26T21:31:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:55 smithi143 ceph-mon[31461]: pgmap v397: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:31:56.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:56.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:56 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2796737318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:31:56.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:56.970 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:56.993 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:31:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:56 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2796737318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:31:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:57.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:56 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2796737318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:31:57.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:57.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.006 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:57 smithi097 ceph-mon[28127]: pgmap v398: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:31:58.006 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.006 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:57 smithi111 ceph-mon[31001]: pgmap v398: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:31:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:57 smithi143 ceph-mon[31461]: pgmap v398: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:31:58.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:58.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:59.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:59 smithi097 ceph-mon[28127]: pgmap v399: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:31:59.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:59.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:59.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:31:59.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:31:59 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:59 smithi143 ceph-mon[31461]: pgmap v399: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:31:59 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:59 smithi111 ceph-mon[31001]: pgmap v399: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:00.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:31:59 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:01.583 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738466 got 34359738472 for osd.0 2024-04-26T21:32:01.584 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:01.766 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:01 smithi097 ceph-mon[28127]: pgmap v400: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:32:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:01 smithi143 ceph-mon[31461]: pgmap v400: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:32:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:01 smithi111 ceph-mon[31001]: pgmap v400: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 -- 172.21.15.97:0/3965356754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 msgr2=0x7f608c10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 --2- 172.21.15.97:0/3965356754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c10f990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6078009990 tx=0x7f6078009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 -- 172.21.15.97:0/3965356754 shutdown_connections 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 --2- 172.21.15.97:0/3965356754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 --2- 172.21.15.97:0/3965356754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 --2- 172.21.15.97:0/3965356754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c10f990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.860+0000 7f6092600700 1 -- 172.21.15.97:0/3965356754 >> 172.21.15.97:0/3965356754 conn(0x7f608c0792d0 msgr2=0x7f608c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.861+0000 7f6092600700 1 -- 172.21.15.97:0/3965356754 shutdown_connections 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.861+0000 7f6092600700 1 -- 172.21.15.97:0/3965356754 wait complete. 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.862+0000 7f6092600700 1 Processor -- start 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 -- start start 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c109060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f608c115750 con 0x7f608c06ba50 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1155d0 con 0x7f608c06b130 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f6092600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1158d0 con 0x7f608c10fed0 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.863+0000 7f608aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34534/0 (socket says 172.21.15.97:34534) 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 -- 172.21.15.97:0/1694493129 learned_addr learned my addr 172.21.15.97:0/1694493129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:03.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 -- 172.21.15.97:0/1694493129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 msgr2=0x7f608c104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608b400700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608be00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 -- 172.21.15.97:0/1694493129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 msgr2=0x7f608c109060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c109060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 -- 172.21.15.97:0/1694493129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c104e30 con 0x7f608c06ba50 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608b400700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608be00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.864+0000 7f608aa00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6078000c00 tx=0x7f60780036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:03.313 INFO:teuthology.orchestra.run.smithi097.stdout:111669149781 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.865+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078027070 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.865+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f607800fc80 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.865+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078022d60 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.865+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c1050c0 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.865+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c10ef50 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.867+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c005330 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.869+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6078022470 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.870+0000 7f6083e00700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 0x7f60700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.870+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f60780d1330 con 0x7f608c06ba50 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.871+0000 7f608b400700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 0x7f60700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.872+0000 7f608b400700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 0x7f60700a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f607c005fd0 tx=0x7f607c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:02.874+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6078013070 con 0x7f608c06ba50 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.105+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f606c0058f0 con 0x7f608c06ba50 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.106+0000 7f6083e00700 1 -- 172.21.15.97:0/1694493129 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f607809dd50 con 0x7f608c06ba50 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 msgr2=0x7f60700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 0x7f60700a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f607c005fd0 tx=0x7f607c005e20 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 msgr2=0x7f608c1095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6078000c00 tx=0x7f60780036f0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 shutdown_connections 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f60700a3d50 0x7f60700a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c06b130 0x7f608c109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f608c10fed0 0x7f608c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 --2- 172.21.15.97:0/1694493129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06ba50 0x7f608c1095a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 >> 172.21.15.97:0/1694493129 conn(0x7f608c0792d0 msgr2=0x7f608c111420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 shutdown_connections 2024-04-26T21:32:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:03.110+0000 7f6092600700 1 -- 172.21.15.97:0/1694493129 wait complete. 2024-04-26T21:32:03.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:03 smithi097 ceph-mon[28127]: pgmap v401: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:03.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:03 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1694493129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:32:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:03 smithi143 ceph-mon[31461]: pgmap v401: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:04.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:03 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1694493129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:32:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:03 smithi111 ceph-mon[31001]: pgmap v401: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:03 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1694493129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 -- 172.21.15.97:0/1218179116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075f70 msgr2=0x7f57c01140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 --2- 172.21.15.97:0/1218179116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075f70 0x7f57c01140b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009990 tx=0x7f57ac009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 -- 172.21.15.97:0/1218179116 shutdown_connections 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 --2- 172.21.15.97:0/1218179116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c0075650 0x7f57c0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 --2- 172.21.15.97:0/1218179116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c01145f0 0x7f57c01169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 --2- 172.21.15.97:0/1218179116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075f70 0x7f57c01140b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 -- 172.21.15.97:0/1218179116 >> 172.21.15.97:0/1218179116 conn(0x7f57c007a580 msgr2=0x7f57c007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.022+0000 7f57c6a00700 1 -- 172.21.15.97:0/1218179116 shutdown_connections 2024-04-26T21:32:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.023+0000 7f57c6a00700 1 -- 172.21.15.97:0/1218179116 wait complete. 2024-04-26T21:32:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.024+0000 7f57c6a00700 1 Processor -- start 2024-04-26T21:32:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.024+0000 7f57c6a00700 1 -- start start 2024-04-26T21:32:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.024+0000 7f57c6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c6a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c0075f70 0x7f57c01a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c6a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 0x7f57c01a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c6a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0119ae0 con 0x7f57c0075650 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c6a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0119960 con 0x7f57c01145f0 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c6a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0119c60 con 0x7f57c0075f70 2024-04-26T21:32:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57c4a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 0x7f57c01a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34592/0 (socket says 172.21.15.97:34592) 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57bfe00700 1 -- 172.21.15.97:0/3513267917 learned_addr learned my addr 172.21.15.97:0/3513267917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.025+0000 7f57bf400700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c0075f70 0x7f57c01a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 -- 172.21.15.97:0/3513267917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c0075f70 msgr2=0x7f57c01a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c0075f70 0x7f57c01a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 -- 172.21.15.97:0/3513267917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 msgr2=0x7f57c01a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 0x7f57c01a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 -- 172.21.15.97:0/3513267917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01a4fe0 con 0x7f57c0075650 2024-04-26T21:32:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57c4a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 0x7f57c01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.026+0000 7f57bfe00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c9b0 tx=0x7f57b000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.027+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0007b00 con 0x7f57c0075650 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.027+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57b000ce50 con 0x7f57c0075650 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.027+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c01a52d0 con 0x7f57c0075650 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.027+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c01acb50 con 0x7f57c0075650 2024-04-26T21:32:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.027+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0018660 con 0x7f57c0075650 2024-04-26T21:32:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.028+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a0005330 con 0x7f57c0075650 2024-04-26T21:32:05.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.030+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f57b0007ca0 con 0x7f57c0075650 2024-04-26T21:32:05.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.032+0000 7f57bcc00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 0x7f57a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.032+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f57b00d0690 con 0x7f57c0075650 2024-04-26T21:32:05.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.032+0000 7f57bf400700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 0x7f57a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.033+0000 7f57bf400700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 0x7f57a40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57ac003760 tx=0x7f57ac00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:05.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.040+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f57b0014030 con 0x7f57c0075650 2024-04-26T21:32:05.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.293+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f57a00058f0 con 0x7f57c0075650 2024-04-26T21:32:05.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.293+0000 7f57bcc00700 1 -- 172.21.15.97:0/3513267917 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f57b009cfb0 con 0x7f57c0075650 2024-04-26T21:32:05.295 INFO:teuthology.orchestra.run.smithi097.stdout:227633266746 2024-04-26T21:32:05.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.296+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 msgr2=0x7f57a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.296+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 0x7f57a40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57ac003760 tx=0x7f57ac00b540 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.296+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 msgr2=0x7f57c019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.296+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c9b0 tx=0x7f57b000ccc0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 shutdown_connections 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f57a40a3cd0 0x7f57a40a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57c01145f0 0x7f57c01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57c0075650 0x7f57c019ffd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 --2- 172.21.15.97:0/3513267917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57c0075f70 0x7f57c01a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 >> 172.21.15.97:0/3513267917 conn(0x7f57c007a580 msgr2=0x7f57c0110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 shutdown_connections 2024-04-26T21:32:05.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.297+0000 7f57c6a00700 1 -- 172.21.15.97:0/3513267917 wait complete. 2024-04-26T21:32:05.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149776 got 111669149781 for osd.3 2024-04-26T21:32:05.743 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:05.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 -- 172.21.15.97:0/315211876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 msgr2=0x7f9f3c109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 --2- 172.21.15.97:0/315211876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c109ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009990 tx=0x7f9f24009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 -- 172.21.15.97:0/315211876 shutdown_connections 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 --2- 172.21.15.97:0/315211876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10a410 0x7f9f3c116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 --2- 172.21.15.97:0/315211876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10fa50 0x7f9f3c10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 --2- 172.21.15.97:0/315211876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c109ed0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 -- 172.21.15.97:0/315211876 >> 172.21.15.97:0/315211876 conn(0x7f9f3c076bf0 msgr2=0x7f9f3c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 -- 172.21.15.97:0/315211876 shutdown_connections 2024-04-26T21:32:05.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.949+0000 7f9f41800700 1 -- 172.21.15.97:0/315211876 wait complete. 2024-04-26T21:32:05.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:05 smithi097 ceph-mon[28127]: pgmap v402: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:05.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:05 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2734101927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:32:05.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:05 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4175943237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:32:05.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:05 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3513267917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.950+0000 7f9f41800700 1 Processor -- start 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.950+0000 7f9f41800700 1 -- start start 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 0x7f9f3c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10fa50 0x7f9f3c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c119b90 con 0x7f9f3c109a50 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c119a10 con 0x7f9f3c10fa50 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f41800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c119d10 con 0x7f9f3c10a410 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f3aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f3aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34608/0 (socket says 172.21.15.97:34608) 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f3aa00700 1 -- 172.21.15.97:0/1172969216 learned_addr learned my addr 172.21.15.97:0/1172969216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.951+0000 7f9f3b400700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10fa50 0x7f9f3c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f33e00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 0x7f9f3c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 -- 172.21.15.97:0/1172969216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 msgr2=0x7f9f3c1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 0x7f9f3c1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 -- 172.21.15.97:0/1172969216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10fa50 msgr2=0x7f9f3c1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10fa50 0x7f9f3c1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 -- 172.21.15.97:0/1172969216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1a5070 con 0x7f9f3c109a50 2024-04-26T21:32:05.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f33e00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 0x7f9f3c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f3aa00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00d980 tx=0x7f9f2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.952+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c017070 con 0x7f9f3c109a50 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.953+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c005020 con 0x7f9f3c109a50 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.953+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c1a5360 con 0x7f9f3c109a50 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.953+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c009ab0 con 0x7f9f3c109a50 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.953+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1acbe0 con 0x7f9f3c109a50 2024-04-26T21:32:05.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.954+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c111fe0 con 0x7f9f3c109a50 2024-04-26T21:32:05.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.956+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f2c0051d0 con 0x7f9f3c109a50 2024-04-26T21:32:05.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.957+0000 7f9f33400700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 0x7f9f200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:05.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.957+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9f2c0d0340 con 0x7f9f3c109a50 2024-04-26T21:32:05.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.958+0000 7f9f33e00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 0x7f9f200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:05.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.958+0000 7f9f33e00700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 0x7f9f200a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f24006010 tx=0x7f9f2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:05.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.962+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f2c020020 con 0x7f9f3c109a50 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 -- 172.21.15.97:0/3603424390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 msgr2=0x7f039c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 --2- 172.21.15.97:0/3603424390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c109880 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f038c009990 tx=0x7f038c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 -- 172.21.15.97:0/3603424390 shutdown_connections 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 --2- 172.21.15.97:0/3603424390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 0x7f039c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 --2- 172.21.15.97:0/3603424390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c109880 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 --2- 172.21.15.97:0/3603424390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 0x7f039c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.929+0000 7f03a3000700 1 -- 172.21.15.97:0/3603424390 >> 172.21.15.97:0/3603424390 conn(0x7f039c0792d0 msgr2=0x7f039c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.930+0000 7f03a3000700 1 -- 172.21.15.97:0/3603424390 shutdown_connections 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.930+0000 7f03a3000700 1 -- 172.21.15.97:0/3603424390 wait complete. 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.932+0000 7f03a3000700 1 Processor -- start 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.932+0000 7f03a3000700 1 -- start start 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 0x7f039c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 0x7f039c1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113460 con 0x7f039c1094a0 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1132e0 con 0x7f039c06bad0 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a3000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1135e0 con 0x7f039c06b130 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a1000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a1000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34574/0 (socket says 172.21.15.97:34574) 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f03a1000700 1 -- 172.21.15.97:0/4175943237 learned_addr learned my addr 172.21.15.97:0/4175943237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f039b400700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 0x7f039c1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:06.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.933+0000 7f039be00700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 0x7f039c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 -- 172.21.15.97:0/4175943237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 msgr2=0x7f039c1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 0x7f039c1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 -- 172.21.15.97:0/4175943237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 msgr2=0x7f039c1a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 0x7f039c1a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 -- 172.21.15.97:0/4175943237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c19c1a0 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.934+0000 7f03a1000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0394007ee0 tx=0x7f039400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.935+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400dbd0 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.935+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f039400dd70 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.935+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c19c490 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.935+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940058d0 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.935+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a8750 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.936+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c06a110 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.938+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f039401a020 con 0x7f039c1094a0 2024-04-26T21:32:06.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.940+0000 7f0398c00700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.940+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f03940cfca0 con 0x7f039c1094a0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.940+0000 7f039be00700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.941+0000 7f039be00700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 0x7f03800a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f038c005850 tx=0x7f038c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.944+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0394010070 con 0x7f039c1094a0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.193+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f039c02e490 con 0x7f039c1094a0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.194+0000 7f0398c00700 1 -- 172.21.15.97:0/4175943237 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f039409c530 con 0x7f039c1094a0 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stdout:55834574949 2024-04-26T21:32:06.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 msgr2=0x7f03800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 0x7f03800a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f038c005850 tx=0x7f038c01a040 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 msgr2=0x7f039c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0394007ee0 tx=0x7f039400d3b0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 shutdown_connections 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f03800a3cd0 0x7f03800a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f039c1094a0 0x7f039c19ba40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f039c06b130 0x7f039c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 --2- 172.21.15.97:0/4175943237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f039c06bad0 0x7f039c1a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 >> 172.21.15.97:0/4175943237 conn(0x7f039c0792d0 msgr2=0x7f039c10e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 shutdown_connections 2024-04-26T21:32:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.198+0000 7f03a3000700 1 -- 172.21.15.97:0/4175943237 wait complete. 2024-04-26T21:32:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:05 smithi143 ceph-mon[31461]: pgmap v402: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:05 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2734101927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:32:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:05 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4175943237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:32:06.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:05 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3513267917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:32:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:05 smithi111 ceph-mon[31001]: pgmap v402: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:05 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2734101927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:32:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:05 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4175943237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:32:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:05 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3513267917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:32:06.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.192+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9f3c1119c0 con 0x7f9f3c109a50 2024-04-26T21:32:06.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.193+0000 7f9f33400700 1 -- 172.21.15.97:0/1172969216 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f2c09cbd0 con 0x7f9f3c109a50 2024-04-26T21:32:06.194 INFO:teuthology.orchestra.run.smithi097.stdout:171798691913 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 msgr2=0x7f9f200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 0x7f9f200a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f24006010 tx=0x7f9f2401a040 comp rx=0 tx=0).stop 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 msgr2=0x7f9f3c1a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00d980 tx=0x7f9f2c00dc90 comp rx=0 tx=0).stop 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 shutdown_connections 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9f200a3d50 0x7f9f200a6210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c10fa50 0x7f9f3c1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f3c109a50 0x7f9f3c1a0060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 --2- 172.21.15.97:0/1172969216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c10a410 0x7f9f3c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:06.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 >> 172.21.15.97:0/1172969216 conn(0x7f9f3c076bf0 msgr2=0x7f9f3c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:06.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 shutdown_connections 2024-04-26T21:32:06.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:06.196+0000 7f9f41800700 1 -- 172.21.15.97:0/1172969216 wait complete. 2024-04-26T21:32:06.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:06 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1172969216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:32:07.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:06 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1172969216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:32:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:06 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1172969216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:32:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 -- 172.21.15.97:0/2918337985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634810b640 msgr2=0x7f634810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 --2- 172.21.15.97:0/2918337985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634810b640 0x7f634810ba20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6338009990 tx=0x7f6338009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 -- 172.21.15.97:0/2918337985 shutdown_connections 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 --2- 172.21.15.97:0/2918337985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6348106000 0x7f6348112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 --2- 172.21.15.97:0/2918337985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634810b640 0x7f634810ba20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 --2- 172.21.15.97:0/2918337985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 0x7f6348105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 -- 172.21.15.97:0/2918337985 >> 172.21.15.97:0/2918337985 conn(0x7f6348101130 msgr2=0x7f6348103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.172+0000 7f634d000700 1 -- 172.21.15.97:0/2918337985 shutdown_connections 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.173+0000 7f634d000700 1 -- 172.21.15.97:0/2918337985 wait complete. 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 Processor -- start 2024-04-26T21:32:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 -- start start 2024-04-26T21:32:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 0x7f634819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634810b640 0x7f63481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63481157e0 con 0x7f6348106000 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6348115660 con 0x7f634810b640 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.174+0000 7f634d000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6348115960 con 0x7f6348105640 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34616/0 (socket says 172.21.15.97:34616) 2024-04-26T21:32:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 -- 172.21.15.97:0/2669337061 learned_addr learned my addr 172.21.15.97:0/2669337061 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:07.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345e00700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 0x7f634819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:07.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6346800700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634810b640 0x7f63481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 -- 172.21.15.97:0/2669337061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 msgr2=0x7f634819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 0x7f634819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 -- 172.21.15.97:0/2669337061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634810b640 msgr2=0x7f63481a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634810b640 0x7f63481a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.175+0000 7f6345400700 1 -- 172.21.15.97:0/2669337061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63481a0d90 con 0x7f6348106000 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f6345400700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f633400b530 tx=0x7f633400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334015020 con 0x7f6348106000 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6334011cc0 con 0x7f6348106000 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63481a1080 con 0x7f6348106000 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334010dc0 con 0x7f6348106000 2024-04-26T21:32:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.176+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a8900 con 0x7f6348106000 2024-04-26T21:32:07.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.178+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6328005330 con 0x7f6348106000 2024-04-26T21:32:07.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.180+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6334018020 con 0x7f6348106000 2024-04-26T21:32:07.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.182+0000 7f633ea00700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 0x7f632c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:07.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.183+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f63340cfa70 con 0x7f6348106000 2024-04-26T21:32:07.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.183+0000 7f6345e00700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 0x7f632c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.183+0000 7f6345e00700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 0x7f632c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6338003760 tx=0x7f633800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:07.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.187+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f633401f020 con 0x7f6348106000 2024-04-26T21:32:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.418+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f63280058f0 con 0x7f6348106000 2024-04-26T21:32:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.419+0000 7f633ea00700 1 -- 172.21.15.97:0/2669337061 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f633409c510 con 0x7f6348106000 2024-04-26T21:32:07.420 INFO:teuthology.orchestra.run.smithi097.stdout:206158430273 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 msgr2=0x7f632c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 0x7f632c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6338003760 tx=0x7f633800b540 comp rx=0 tx=0).stop 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 msgr2=0x7f634819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f633400b530 tx=0x7f633400b840 comp rx=0 tx=0).stop 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 shutdown_connections 2024-04-26T21:32:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f632c0a3d50 0x7f632c0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634810b640 0x7f63481a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6348105640 0x7f634819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 --2- 172.21.15.97:0/2669337061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6348106000 0x7f634819c2c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 >> 172.21.15.97:0/2669337061 conn(0x7f6348101130 msgr2=0x7f6348101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 shutdown_connections 2024-04-26T21:32:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:07.422+0000 7f634d000700 1 -- 172.21.15.97:0/2669337061 wait complete. 2024-04-26T21:32:07.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266740 got 227633266746 for osd.7 2024-04-26T21:32:07.615 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:08.008 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:07 smithi097 ceph-mon[28127]: pgmap v403: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:08.008 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2669337061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:32:08.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:07 smithi143 ceph-mon[31461]: pgmap v403: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2669337061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:32:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:07 smithi111 ceph-mon[31001]: pgmap v403: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2669337061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:32:08.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430267 got 206158430273 for osd.6 2024-04-26T21:32:08.439 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:09.438 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:09.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 -- 172.21.15.97:0/3466336485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc102ce0 msgr2=0x7ff1fc10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 --2- 172.21.15.97:0/3466336485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc10f1d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b0a0 tx=0x7ff1f400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 -- 172.21.15.97:0/3466336485 shutdown_connections 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 --2- 172.21.15.97:0/3466336485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc10f1d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 --2- 172.21.15.97:0/3466336485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc108390 0x7ff1fc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 --2- 172.21.15.97:0/3466336485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102320 0x7ff1fc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 -- 172.21.15.97:0/3466336485 >> 172.21.15.97:0/3466336485 conn(0x7ff1fc0792d0 msgr2=0x7ff1fc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 -- 172.21.15.97:0/3466336485 shutdown_connections 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.776+0000 7ff202400700 1 -- 172.21.15.97:0/3466336485 wait complete. 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 Processor -- start 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 -- start start 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc102320 0x7ff1fc1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc06b5d0 con 0x7ff1fc108390 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc06b450 con 0x7ff1fc102ce0 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.777+0000 7ff202400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc06b750 con 0x7ff1fc102320 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34560/0 (socket says 172.21.15.97:34560) 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 -- 172.21.15.97:0/2734101927 learned_addr learned my addr 172.21.15.97:0/2734101927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fb400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc102320 0x7ff1fc1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1faa00700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 -- 172.21.15.97:0/2734101927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc102320 msgr2=0x7ff1fc1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc102320 0x7ff1fc1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 -- 172.21.15.97:0/2734101927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102ce0 msgr2=0x7ff1fc1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc1a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 -- 172.21.15.97:0/2734101927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc19c160 con 0x7ff1fc108390 2024-04-26T21:32:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.778+0000 7ff1fbe00700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b070 tx=0x7ff1f4004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.779+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4021070 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.779+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f4007c00 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.779+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4004780 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.779+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc19c3f0 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.779+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a8710 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.781+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc103a40 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.784+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1f4003a00 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.786+0000 7ff1f3e00700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 0x7ff1e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.787+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff1f40d1610 con 0x7ff1fc108390 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.787+0000 7ff1fb400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 0x7ff1e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:09.643 INFO:teuthology.orchestra.run.smithi097.stdout:266287972404 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.787+0000 7ff1fb400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 0x7ff1e00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005e10 tx=0x7ff1ec005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:04.791+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff1f409df30 con 0x7ff1fc108390 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.038+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff1fc1a88e0 con 0x7ff1fc108390 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.039+0000 7ff1f3e00700 1 -- 172.21.15.97:0/2734101927 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1f409d830 con 0x7ff1fc108390 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.042+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 msgr2=0x7ff1e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.042+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 0x7ff1e00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005e10 tx=0x7ff1ec005d40 comp rx=0 tx=0).stop 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.042+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 msgr2=0x7ff1fc19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.042+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b070 tx=0x7ff1f4004e00 comp rx=0 tx=0).stop 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 shutdown_connections 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1e00a3d50 0x7ff1e00a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1fc108390 0x7ff1fc19ba00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1fc102320 0x7ff1fc1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 --2- 172.21.15.97:0/2734101927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1fc102ce0 0x7ff1fc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 >> 172.21.15.97:0/2734101927 conn(0x7ff1fc0792d0 msgr2=0x7ff1fc10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 shutdown_connections 2024-04-26T21:32:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:05.043+0000 7ff202400700 1 -- 172.21.15.97:0/2734101927 wait complete. 2024-04-26T21:32:09.990 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:09 smithi097 ceph-mon[28127]: pgmap v404: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:10.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:09 smithi143 ceph-mon[31461]: pgmap v404: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:09 smithi111 ceph-mon[31001]: pgmap v404: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:10.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:10 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/682831723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:32:11.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:10 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/682831723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:32:11.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:10 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/682831723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:32:11.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972399 got 266287972404 for osd.8 2024-04-26T21:32:11.311 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:11.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691908 got 171798691913 for osd.5 2024-04-26T21:32:11.325 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:11.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 -- 172.21.15.97:0/3416110969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c107590 msgr2=0x7faf5c10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 --2- 172.21.15.97:0/3416110969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c107590 0x7faf5c10f160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf5400b0d0 tx=0x7faf5400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 -- 172.21.15.97:0/3416110969 shutdown_connections 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 --2- 172.21.15.97:0/3416110969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c107590 0x7faf5c10f160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 --2- 172.21.15.97:0/3416110969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c1024b0 0x7faf5c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 --2- 172.21.15.97:0/3416110969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102e60 0x7faf5c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 -- 172.21.15.97:0/3416110969 >> 172.21.15.97:0/3416110969 conn(0x7faf5c0fdd40 msgr2=0x7faf5c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 -- 172.21.15.97:0/3416110969 shutdown_connections 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.268+0000 7faf62000700 1 -- 172.21.15.97:0/3416110969 wait complete. 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.269+0000 7faf62000700 1 Processor -- start 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 -- start start 2024-04-26T21:32:11.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1024b0 0x7faf5c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c107590 0x7faf5c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c06b9e0 con 0x7faf5c102e60 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c06b860 con 0x7faf5c1024b0 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.270+0000 7faf62000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c06bb60 con 0x7faf5c107590 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34626/0 (socket says 172.21.15.97:34626) 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 -- 172.21.15.97:0/682831723 learned_addr learned my addr 172.21.15.97:0/682831723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf60c00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1024b0 0x7faf5c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf61600700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c107590 0x7faf5c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 -- 172.21.15.97:0/682831723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c107590 msgr2=0x7faf5c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c107590 0x7faf5c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 -- 172.21.15.97:0/682831723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1024b0 msgr2=0x7faf5c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1024b0 0x7faf5c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.271+0000 7faf5be00700 1 -- 172.21.15.97:0/682831723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5c19c060 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf5be00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d980 tx=0x7faf4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48017070 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf4800fd20 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800e3b0 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5c19c350 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.272+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5c1a8610 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.273+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5c06a110 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.276+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf4800f890 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.278+0000 7faf59600700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 0x7faf440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.278+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7faf480cfec0 con 0x7faf5c102e60 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.278+0000 7faf60c00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 0x7faf440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.279+0000 7faf60c00700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 0x7faf440a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf50005950 tx=0x7faf5000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:11.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.287+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faf4809c750 con 0x7faf5c102e60 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stdout:300647710765 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.511+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faf5c1a8840 con 0x7faf5c102e60 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.512+0000 7faf59600700 1 -- 172.21.15.97:0/682831723 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faf4809c050 con 0x7faf5c102e60 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.514+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 msgr2=0x7faf440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.514+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 0x7faf440a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf50005950 tx=0x7faf5000a300 comp rx=0 tx=0).stop 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.514+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 msgr2=0x7faf5c1a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.514+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d980 tx=0x7faf4800dc90 comp rx=0 tx=0).stop 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 shutdown_connections 2024-04-26T21:32:11.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faf440a3d50 0x7faf440a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1024b0 0x7faf5c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf5c107590 0x7faf5c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 --2- 172.21.15.97:0/682831723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf5c102e60 0x7faf5c1a1c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 >> 172.21.15.97:0/682831723 conn(0x7faf5c0fdd40 msgr2=0x7faf5c10d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 shutdown_connections 2024-04-26T21:32:11.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:10.515+0000 7faf62000700 1 -- 172.21.15.97:0/682831723 wait complete. 2024-04-26T21:32:12.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:11 smithi143 ceph-mon[31461]: pgmap v405: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:12.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:11 smithi097 ceph-mon[28127]: pgmap v405: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:11 smithi111 ceph-mon[31001]: pgmap v405: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:12.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574943 got 55834574949 for osd.1 2024-04-26T21:32:12.725 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 -- 172.21.15.97:0/964741016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 msgr2=0x7f6e00113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 --2- 172.21.15.97:0/964741016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e00113e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009990 tx=0x7f6dec009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 -- 172.21.15.97:0/964741016 shutdown_connections 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 --2- 172.21.15.97:0/964741016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e00114350 0x7f6e00116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 --2- 172.21.15.97:0/964741016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e000757c0 0x7f6e00075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 --2- 172.21.15.97:0/964741016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e00113e10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 -- 172.21.15.97:0/964741016 >> 172.21.15.97:0/964741016 conn(0x7f6e000fee30 msgr2=0x7f6e00101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.932+0000 7f6dffe00700 1 -- 172.21.15.97:0/964741016 shutdown_connections 2024-04-26T21:32:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.933+0000 7f6dffe00700 1 -- 172.21.15.97:0/964741016 wait complete. 2024-04-26T21:32:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 Processor -- start 2024-04-26T21:32:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 -- start start 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e000757c0 0x7f6e001a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e00114350 0x7f6e001a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00119aa0 con 0x7f6e000760e0 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00119920 con 0x7f6e000757c0 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dffe00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00119c20 con 0x7f6e00114350 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dfe000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dfe000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44726/0 (socket says 172.21.15.97:44726) 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.934+0000 7f6dfe000700 1 -- 172.21.15.97:0/4118412280 learned_addr learned my addr 172.21.15.97:0/4118412280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfea00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e000757c0 0x7f6e001a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dff400700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e00114350 0x7f6e001a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 -- 172.21.15.97:0/4118412280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e00114350 msgr2=0x7f6e001a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e00114350 0x7f6e001a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 -- 172.21.15.97:0/4118412280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e000757c0 msgr2=0x7f6e001a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e000757c0 0x7f6e001a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 -- 172.21.15.97:0/4118412280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e001a5080 con 0x7f6e000760e0 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.935+0000 7f6dfe000700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005910 tx=0x7f6dec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.936+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec027070 con 0x7f6e000760e0 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.936+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dec00fce0 con 0x7f6e000760e0 2024-04-26T21:32:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.936+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e001a5310 con 0x7f6e000760e0 2024-04-26T21:32:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.936+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0173f0 con 0x7f6e000760e0 2024-04-26T21:32:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.936+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e001acbf0 con 0x7f6e000760e0 2024-04-26T21:32:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.937+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0006a110 con 0x7f6e000760e0 2024-04-26T21:32:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.939+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6dec02a070 con 0x7f6e000760e0 2024-04-26T21:32:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.940+0000 7f6df7400700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 0x7f6de40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.940+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f6dec0d1ce0 con 0x7f6e000760e0 2024-04-26T21:32:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.940+0000 7f6dfea00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 0x7f6de40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.941+0000 7f6dfea00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 0x7f6de40a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006fd0 tx=0x7f6df0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:12.943+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6dec031070 con 0x7f6e000760e0 2024-04-26T21:32:13.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.182+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6e0002e490 con 0x7f6e000760e0 2024-04-26T21:32:13.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.183+0000 7f6df7400700 1 -- 172.21.15.97:0/4118412280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6dec013070 con 0x7f6e000760e0 2024-04-26T21:32:13.184 INFO:teuthology.orchestra.run.smithi097.stdout:146028888144 2024-04-26T21:32:13.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.185+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 msgr2=0x7f6de40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.185+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 0x7f6de40a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006fd0 tx=0x7f6df0008040 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.185+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 msgr2=0x7f6e001a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.185+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005910 tx=0x7f6dec009f90 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 shutdown_connections 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6de40a3d50 0x7f6de40a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e000757c0 0x7f6e001a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e00114350 0x7f6e001a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 --2- 172.21.15.97:0/4118412280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e000760e0 0x7f6e001a05b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 >> 172.21.15.97:0/4118412280 conn(0x7f6e000fee30 msgr2=0x7f6e001007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 shutdown_connections 2024-04-26T21:32:13.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:13.186+0000 7f6dffe00700 1 -- 172.21.15.97:0/4118412280 wait complete. 2024-04-26T21:32:13.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:13 smithi097 ceph-mon[28127]: pgmap v406: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:13.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:13 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4118412280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:32:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:13 smithi143 ceph-mon[31461]: pgmap v406: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:14.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:13 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4118412280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:32:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:13 smithi111 ceph-mon[31001]: pgmap v406: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:14.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:13 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4118412280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:32:14.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710761 got 300647710765 for osd.9 2024-04-26T21:32:14.192 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:15.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888136 got 146028888144 for osd.4 2024-04-26T21:32:15.884 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:16.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:15 smithi143 ceph-mon[31461]: pgmap v407: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:16.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:15 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/805504758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:32:16.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:15 smithi097 ceph-mon[28127]: pgmap v407: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:16.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:15 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/805504758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:32:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:15 smithi111 ceph-mon[31001]: pgmap v407: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:15 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/805504758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:32:16.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 -- 172.21.15.97:0/51279733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105610 msgr2=0x7f0344105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:16.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 --2- 172.21.15.97:0/51279733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105610 0x7f0344105a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0330009990 tx=0x7f0330009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 -- 172.21.15.97:0/51279733 shutdown_connections 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 --2- 172.21.15.97:0/51279733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f034410b6a0 0x7f034410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 --2- 172.21.15.97:0/51279733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0344105fd0 0x7f03441124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 --2- 172.21.15.97:0/51279733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105610 0x7f0344105a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 -- 172.21.15.97:0/51279733 >> 172.21.15.97:0/51279733 conn(0x7f0344101100 msgr2=0x7f0344103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 -- 172.21.15.97:0/51279733 shutdown_connections 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.229+0000 7f034c000700 1 -- 172.21.15.97:0/51279733 wait complete. 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.230+0000 7f034c000700 1 Processor -- start 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.230+0000 7f034c000700 1 -- start start 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 0x7f03441a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 0x7f034419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0344115700 con 0x7f0344105fd0 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0344115580 con 0x7f0344105610 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034c000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0344115880 con 0x7f034410b6a0 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f0348c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f0348c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44746/0 (socket says 172.21.15.97:44746) 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f0348c00700 1 -- 172.21.15.97:0/805504758 learned_addr learned my addr 172.21.15.97:0/805504758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f0349600700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 0x7f03441a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.231+0000 7f034a000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 0x7f034419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 -- 172.21.15.97:0/805504758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 msgr2=0x7f034419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 0x7f034419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 -- 172.21.15.97:0/805504758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 msgr2=0x7f03441a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:16.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 0x7f03441a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 -- 172.21.15.97:0/805504758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034419c100 con 0x7f0344105fd0 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0349600700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 0x7f03441a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f034a000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 0x7f034419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0348c00700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f03300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330027070 con 0x7f0344105fd0 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f033000fdb0 con 0x7f0344105fd0 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300173f0 con 0x7f0344105fd0 2024-04-26T21:32:16.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034419c390 con 0x7f0344105fd0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.232+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a86b0 con 0x7f0344105fd0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.233+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034406a110 con 0x7f0344105fd0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.235+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f033002a070 con 0x7f0344105fd0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.237+0000 7f0342000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 0x7f032c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.237+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f03300d1a50 con 0x7f0344105fd0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.237+0000 7f0349600700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 0x7f032c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.238+0000 7f0349600700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 0x7f032c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0338005fd0 tx=0x7f0338005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:16.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.239+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0330031070 con 0x7f0344105fd0 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stdout:322122547239 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.473+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f034402e490 con 0x7f0344105fd0 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.474+0000 7f0342000700 1 -- 172.21.15.97:0/805504758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f03300a4030 con 0x7f0344105fd0 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 msgr2=0x7f032c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 0x7f032c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0338005fd0 tx=0x7f0338005e60 comp rx=0 tx=0).stop 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 msgr2=0x7f03441a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:16.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f03300047b0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 shutdown_connections 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f032c0a3d50 0x7f032c0a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0344105610 0x7f03441a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034410b6a0 0x7f034419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 --2- 172.21.15.97:0/805504758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344105fd0 0x7f03441a1d20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 >> 172.21.15.97:0/805504758 conn(0x7f0344101100 msgr2=0x7f0344102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.478+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 shutdown_connections 2024-04-26T21:32:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:15.479+0000 7f034c000700 1 -- 172.21.15.97:0/805504758 wait complete. 2024-04-26T21:32:17.063 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547232 got 322122547239 for osd.10 2024-04-26T21:32:17.063 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:17.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 -- 172.21.15.97:0/4199565916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 msgr2=0x7f99041140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:17.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 --2- 172.21.15.97:0/4199565916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041140b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009990 tx=0x7f98ec009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 -- 172.21.15.97:0/4199565916 shutdown_connections 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 --2- 172.21.15.97:0/4199565916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9904075650 0x7f9904075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 --2- 172.21.15.97:0/4199565916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99041145f0 0x7f99041169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 --2- 172.21.15.97:0/4199565916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041140b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 -- 172.21.15.97:0/4199565916 >> 172.21.15.97:0/4199565916 conn(0x7f990407a580 msgr2=0x7f990407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 -- 172.21.15.97:0/4199565916 shutdown_connections 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.827+0000 7f990a200700 1 -- 172.21.15.97:0/4199565916 wait complete. 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.828+0000 7f990a200700 1 Processor -- start 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 -- start start 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9904075650 0x7f99041a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99041145f0 0x7f99041a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:17.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9904119ae0 con 0x7f9904075f70 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9904119960 con 0x7f99041145f0 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f990a200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9904119c60 con 0x7f9904075650 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f98faa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f98faa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44758/0 (socket says 172.21.15.97:44758) 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.829+0000 7f98faa00700 1 -- 172.21.15.97:0/2037478138 learned_addr learned my addr 172.21.15.97:0/2037478138 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f9903400700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9904075650 0x7f99041a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f9903e00700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99041145f0 0x7f99041a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 -- 172.21.15.97:0/2037478138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9904075650 msgr2=0x7f99041a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9904075650 0x7f99041a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 -- 172.21.15.97:0/2037478138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99041145f0 msgr2=0x7f99041a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99041145f0 0x7f99041a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 -- 172.21.15.97:0/2037478138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99041a5060 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.830+0000 7f98faa00700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005910 tx=0x7f98ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.831+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec027070 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.831+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98ec00fc00 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.831+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec022d40 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.831+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99041a52f0 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.831+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041acbd0 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.832+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990406a110 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.834+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f98ec02a070 con 0x7f9904075f70 2024-04-26T21:32:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.835+0000 7f9900c00700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 0x7f98e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:17.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.835+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f98ec0d1300 con 0x7f9904075f70 2024-04-26T21:32:17.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.836+0000 7f9903400700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 0x7f98e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:17.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.836+0000 7f9903400700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 0x7f98e80a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009d10 tx=0x7f98f4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:17.612 INFO:teuthology.orchestra.run.smithi097.stdout:356482285601 2024-04-26T21:32:17.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:16.840+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f98ec05f060 con 0x7f9904075f70 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.082+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f99041acda0 con 0x7f9904075f70 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.083+0000 7f9900c00700 1 -- 172.21.15.97:0/2037478138 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f98ec013070 con 0x7f9904075f70 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 msgr2=0x7f98e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 0x7f98e80a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009d10 tx=0x7f98f4009040 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 msgr2=0x7f99041a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005910 tx=0x7f98ec0047b0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 shutdown_connections 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f98e80a3cd0 0x7f98e80a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99041145f0 0x7f99041a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9904075650 0x7f99041a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 --2- 172.21.15.97:0/2037478138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9904075f70 0x7f99041a0590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 >> 172.21.15.97:0/2037478138 conn(0x7f990407a580 msgr2=0x7f9904110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 shutdown_connections 2024-04-26T21:32:17.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:17.086+0000 7f990a200700 1 -- 172.21.15.97:0/2037478138 wait complete. 2024-04-26T21:32:18.026 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:17 smithi097 ceph-mon[28127]: pgmap v408: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:18.026 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:17 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2037478138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:32:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:17 smithi143 ceph-mon[31461]: pgmap v408: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:18.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:17 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2037478138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:32:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:17 smithi111 ceph-mon[31001]: pgmap v408: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:17 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2037478138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:32:18.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 -- 172.21.15.97:0/1216549335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105f30 msgr2=0x7f302c1123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:18.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 --2- 172.21.15.97:0/1216549335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105f30 0x7f302c1123f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f302400b0d0 tx=0x7f302400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 -- 172.21.15.97:0/1216549335 shutdown_connections 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 --2- 172.21.15.97:0/1216549335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 0x7f302c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 --2- 172.21.15.97:0/1216549335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105f30 0x7f302c1123f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 --2- 172.21.15.97:0/1216549335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105570 0x7f302c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 -- 172.21.15.97:0/1216549335 >> 172.21.15.97:0/1216549335 conn(0x7f302c101040 msgr2=0x7f302c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 -- 172.21.15.97:0/1216549335 shutdown_connections 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.345+0000 7f3030a00700 1 -- 172.21.15.97:0/1216549335 wait complete. 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 Processor -- start 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 -- start start 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105f30 0x7f302c1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 0x7f302c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f302c115640 con 0x7f302c105570 2024-04-26T21:32:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1154c0 con 0x7f302c10b620 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.347+0000 7f3030a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1157c0 con 0x7f302c105f30 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44788/0 (socket says 172.21.15.97:44788) 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 -- 172.21.15.97:0/3710030922 learned_addr learned my addr 172.21.15.97:0/3710030922 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f3023e00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105f30 0x7f302c1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302be00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 0x7f302c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 -- 172.21.15.97:0/3710030922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105f30 msgr2=0x7f302c1a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105f30 0x7f302c1a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 -- 172.21.15.97:0/3710030922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 msgr2=0x7f302c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 0x7f302c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 -- 172.21.15.97:0/3710030922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c19c040 con 0x7f302c105570 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.348+0000 7f302b400700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f301c00d950 tx=0x7f301c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.349+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c016070 con 0x7f302c105570 2024-04-26T21:32:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.349+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f301c010460 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.349+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c19c330 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.349+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c00f620 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.349+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1a85a0 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.350+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c06a110 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.352+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f301c019070 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.353+0000 7f3028c00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 0x7f30100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.353+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f301c0d0170 con 0x7f302c105570 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.354+0000 7f3023e00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 0x7f30100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.354+0000 7f3023e00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 0x7f30100a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3014005950 tx=0x7f30140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.358+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f301c09ca00 con 0x7f302c105570 2024-04-26T21:32:18.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.584+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f302c02e490 con 0x7f302c105570 2024-04-26T21:32:18.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.585+0000 7f3028c00700 1 -- 172.21.15.97:0/3710030922 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f301c09c300 con 0x7f302c105570 2024-04-26T21:32:18.586 INFO:teuthology.orchestra.run.smithi097.stdout:77309411424 2024-04-26T21:32:18.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 msgr2=0x7f30100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:18.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 0x7f30100a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3014005950 tx=0x7f30140058e0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 msgr2=0x7f302c1a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:18.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f301c00d950 tx=0x7f301c00dd10 comp rx=0 tx=0).stop 2024-04-26T21:32:18.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 shutdown_connections 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f30100a3cd0 0x7f30100a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f302c10b620 0x7f302c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f302c105570 0x7f302c1a16d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 --2- 172.21.15.97:0/3710030922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c105f30 0x7f302c1a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 >> 172.21.15.97:0/3710030922 conn(0x7f302c101040 msgr2=0x7f302c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 shutdown_connections 2024-04-26T21:32:18.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:18.588+0000 7f3030a00700 1 -- 172.21.15.97:0/3710030922 wait complete. 2024-04-26T21:32:19.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:18 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3710030922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:32:19.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:18 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3710030922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:32:19.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:18 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3710030922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:32:19.359 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285595 got 356482285601 for osd.11 2024-04-26T21:32:19.359 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:19.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411419 got 77309411424 for osd.2 2024-04-26T21:32:19.738 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:19.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T21:32:19.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:32:20.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:19 smithi143 ceph-mon[31461]: pgmap v409: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:19 smithi097 ceph-mon[28127]: pgmap v409: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:19 smithi111 ceph-mon[31001]: pgmap v409: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:21 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:21.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:21 smithi097 ceph-mon[28127]: pgmap v410: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:21.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:21.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:21 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:21.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:21 smithi111 ceph-mon[31001]: pgmap v410: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:22.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:21 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:22.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:21 smithi143 ceph-mon[31461]: pgmap v410: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:22.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.401+0000 7f8dd3c00700 1 -- 172.21.15.97:0/3284675835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc10b630 msgr2=0x7f8dcc10ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:22.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.401+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/3284675835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc10b630 0x7f8dcc10ba10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.401+0000 7f8dd3c00700 1 -- 172.21.15.97:0/3284675835 shutdown_connections 2024-04-26T21:32:22.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/3284675835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 0x7f8dcc112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/3284675835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc10b630 0x7f8dcc10ba10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/3284675835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc105630 0x7f8dcc105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 -- 172.21.15.97:0/3284675835 >> 172.21.15.97:0/3284675835 conn(0x7f8dcc101160 msgr2=0x7f8dcc103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:22.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 -- 172.21.15.97:0/3284675835 shutdown_connections 2024-04-26T21:32:22.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.402+0000 7f8dd3c00700 1 -- 172.21.15.97:0/3284675835 wait complete. 2024-04-26T21:32:22.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.403+0000 7f8dd3c00700 1 Processor -- start 2024-04-26T21:32:22.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.403+0000 7f8dd3c00700 1 -- start start 2024-04-26T21:32:22.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 0x7f8dcc077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc10b630 0x7f8dcc0783b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc115710 con 0x7f8dcc105630 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc115590 con 0x7f8dcc10b630 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd3c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc115890 con 0x7f8dcc105ff0 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59686/0 (socket says 172.21.15.97:59686) 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 -- 172.21.15.97:0/2741029291 learned_addr learned my addr 172.21.15.97:0/2741029291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dcbe00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 0x7f8dcc077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc10b630 0x7f8dcc0783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 -- 172.21.15.97:0/2741029291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 msgr2=0x7f8dcc077e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:22.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 0x7f8dcc077e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 -- 172.21.15.97:0/2741029291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc10b630 msgr2=0x7f8dcc0783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc10b630 0x7f8dcc0783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.404+0000 7f8dd1200700 1 -- 172.21.15.97:0/2741029291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc1ac330 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dd1200700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000c00 tx=0x7f8dc00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0027070 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc000fb00 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0022b70 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc1ac5c0 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.405+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc1ac9c0 con 0x7f8dcc105630 2024-04-26T21:32:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.407+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc106d70 con 0x7f8dcc105630 2024-04-26T21:32:22.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.409+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8dc0022d10 con 0x7f8dcc105630 2024-04-26T21:32:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.410+0000 7f8dc9600700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 0x7f8db40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.410+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8dc00d1460 con 0x7f8dcc105630 2024-04-26T21:32:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.410+0000 7f8dcbe00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 0x7f8db40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.411+0000 7f8dcbe00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 0x7f8db40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0790f0 tx=0x7f8db800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.415+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8dc005f060 con 0x7f8dcc105630 2024-04-26T21:32:22.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.642+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8dcc063860 con 0x7f8db40a3d50 2024-04-26T21:32:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.666+0000 7f8dc9600700 1 -- 172.21.15.97:0/2741029291 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f8dcc063860 con 0x7f8db40a3d50 2024-04-26T21:32:22.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:22 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:22.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:22 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:22.667 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:22 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:22.672 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 msgr2=0x7f8db40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 0x7f8db40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0790f0 tx=0x7f8db800b410 comp rx=0 tx=0).stop 2024-04-26T21:32:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 msgr2=0x7f8dcc077930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000c00 tx=0x7f8dc00057a0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 shutdown_connections 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8db40a3d50 0x7f8db40a6210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8dcc10b630 0x7f8dcc0783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dcc105630 0x7f8dcc077930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 --2- 172.21.15.97:0/2741029291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8dcc105ff0 0x7f8dcc077e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.672+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 >> 172.21.15.97:0/2741029291 conn(0x7f8dcc101160 msgr2=0x7f8dcc101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.673+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 shutdown_connections 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:22.673+0000 7f8dd3c00700 1 -- 172.21.15.97:0/2741029291 wait complete. 2024-04-26T21:32:22.675 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-26T21:32:22.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:22 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:22.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:22 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:22.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:22 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:23.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:22 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:23.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:22 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:23.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:22 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:32:23.486 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-04-26T21:32:20.685337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650441,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":935,"num_read_kb":578,"num_write":419,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392004,"kb_used_data":13668,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549820,"statfs":{"total":1151940427776,"available":1151539015680,"internally_reserved":0,"allocated":13996032,"data_stored":7575591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005037"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.838455+0000","last_change":"2024-04-26T21:30:53.833872+0000","last_active":"2024-04-26T21:31:21.838455+0000","last_peered":"2024-04-26T21:31:21.838455+0000","last_clean":"2024-04-26T21:31:21.838455+0000","last_became_active":"2024-04-26T21:30:53.832109+0000","last_became_peered":"2024-04-26T21:30:53.832109+0000","last_unstale":"2024-04-26T21:31:21.838455+0000","last_undegraded":"2024-04-26T21:31:21.838455+0000","last_fullsized":"2024-04-26T21:31:21.838455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:50.076505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844785+0000","last_change":"2024-04-26T21:30:53.827034+0000","last_active":"2024-04-26T21:31:21.844785+0000","last_peered":"2024-04-26T21:31:21.844785+0000","last_clean":"2024-04-26T21:31:21.844785+0000","last_became_active":"2024-04-26T21:30:53.826682+0000","last_became_peered":"2024-04-26T21:30:53.826682+0000","last_unstale":"2024-04-26T21:31:21.844785+0000","last_undegraded":"2024-04-26T21:31:21.844785+0000","last_fullsized":"2024-04-26T21:31:21.844785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:39.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844320+0000","last_change":"2024-04-26T21:30:53.833196+0000","last_active":"2024-04-26T21:31:21.844320+0000","last_peered":"2024-04-26T21:31:21.844320+0000","last_clean":"2024-04-26T21:31:21.844320+0000","last_became_active":"2024-04-26T21:30:53.831214+0000","last_became_peered":"2024-04-26T21:30:53.831214+0000","last_unstale":"2024-04-26T21:31:21.844320+0000","last_undegraded":"2024-04-26T21:31:21.844320+0000","last_fullsized":"2024-04-26T21:31:21.844320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:17.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.082060+0000","last_change":"2024-04-26T21:30:53.829249+0000","last_active":"2024-04-26T21:31:22.082060+0000","last_peered":"2024-04-26T21:31:22.082060+0000","last_clean":"2024-04-26T21:31:22.082060+0000","last_became_active":"2024-04-26T21:30:53.828170+0000","last_became_peered":"2024-04-26T21:30:53.828170+0000","last_unstale":"2024-04-26T21:31:22.082060+0000","last_undegraded":"2024-04-26T21:31:22.082060+0000","last_fullsized":"2024-04-26T21:31:22.082060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:30.729567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.847479+0000","last_change":"2024-04-26T21:30:53.824137+0000","last_active":"2024-04-26T21:31:21.847479+0000","last_peered":"2024-04-26T21:31:21.847479+0000","last_clean":"2024-04-26T21:31:21.847479+0000","last_became_active":"2024-04-26T21:30:53.822807+0000","last_became_peered":"2024-04-26T21:30:53.822807+0000","last_unstale":"2024-04-26T21:31:21.847479+0000","last_undegraded":"2024-04-26T21:31:21.847479+0000","last_fullsized":"2024-04-26T21:31:21.847479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:03.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.947619+0000","last_change":"2024-04-26T21:30:53.835932+0000","last_active":"2024-04-26T21:31:21.947619+0000","last_peered":"2024-04-26T21:31:21.947619+0000","last_clean":"2024-04-26T21:31:21.947619+0000","last_became_active":"2024-04-26T21:30:53.835500+0000","last_became_peered":"2024-04-26T21:30:53.835500+0000","last_unstale":"2024-04-26T21:31:21.947619+0000","last_undegraded":"2024-04-26T21:31:21.947619+0000","last_fullsized":"2024-04-26T21:31:21.947619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:34.312999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519509+0000","last_change":"2024-04-26T21:30:53.815175+0000","last_active":"2024-04-26T21:31:22.519509+0000","last_peered":"2024-04-26T21:31:22.519509+0000","last_clean":"2024-04-26T21:31:22.519509+0000","last_became_active":"2024-04-26T21:30:53.814774+0000","last_became_peered":"2024-04-26T21:30:53.814774+0000","last_unstale":"2024-04-26T21:31:22.519509+0000","last_undegraded":"2024-04-26T21:31:22.519509+0000","last_fullsized":"2024-04-26T21:31:22.519509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:21.815872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.841837+0000","last_change":"2024-04-26T21:30:53.824718+0000","last_active":"2024-04-26T21:31:21.841837+0000","last_peered":"2024-04-26T21:31:21.841837+0000","last_clean":"2024-04-26T21:31:21.841837+0000","last_became_active":"2024-04-26T21:30:53.824372+0000","last_became_peered":"2024-04-26T21:30:53.824372+0000","last_unstale":"2024-04-26T21:31:21.841837+0000","last_undegraded":"2024-04-26T21:31:21.841837+0000","last_fullsized":"2024-04-26T21:31:21.841837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:33.450906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831963+0000","last_change":"2024-04-26T21:30:53.830237+0000","last_active":"2024-04-26T21:31:21.831963+0000","last_peered":"2024-04-26T21:31:21.831963+0000","last_clean":"2024-04-26T21:31:21.831963+0000","last_became_active":"2024-04-26T21:30:53.829837+0000","last_became_peered":"2024-04-26T21:30:53.829837+0000","last_unstale":"2024-04-26T21:31:21.831963+0000","last_undegraded":"2024-04-26T21:31:21.831963+0000","last_fullsized":"2024-04-26T21:31:21.831963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:33:15.161046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948554+0000","last_change":"2024-04-26T21:30:53.838240+0000","last_active":"2024-04-26T21:31:21.948554+0000","last_peered":"2024-04-26T21:31:21.948554+0000","last_clean":"2024-04-26T21:31:21.948554+0000","last_became_active":"2024-04-26T21:30:53.837741+0000","last_became_peered":"2024-04-26T21:30:53.837741+0000","last_unstale":"2024-04-26T21:31:21.948554+0000","last_undegraded":"2024-04-26T21:31:21.948554+0000","last_fullsized":"2024-04-26T21:31:21.948554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:33.879043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520424+0000","last_change":"2024-04-26T21:30:53.830564+0000","last_active":"2024-04-26T21:31:22.520424+0000","last_peered":"2024-04-26T21:31:22.520424+0000","last_clean":"2024-04-26T21:31:22.520424+0000","last_became_active":"2024-04-26T21:30:53.829967+0000","last_became_peered":"2024-04-26T21:30:53.829967+0000","last_unstale":"2024-04-26T21:31:22.520424+0000","last_undegraded":"2024-04-26T21:31:22.520424+0000","last_fullsized":"2024-04-26T21:31:22.520424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:06.931219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.081769+0000","last_change":"2024-04-26T21:30:53.829434+0000","last_active":"2024-04-26T21:31:22.081769+0000","last_peered":"2024-04-26T21:31:22.081769+0000","last_clean":"2024-04-26T21:31:22.081769+0000","last_became_active":"2024-04-26T21:30:53.828657+0000","last_became_peered":"2024-04-26T21:30:53.828657+0000","last_unstale":"2024-04-26T21:31:22.081769+0000","last_undegraded":"2024-04-26T21:31:22.081769+0000","last_fullsized":"2024-04-26T21:31:22.081769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:12.215856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.035994+0000","last_change":"2024-04-26T21:31:22.035994+0000","last_active":"2024-04-26T21:31:22.035994+0000","last_peered":"2024-04-26T21:31:22.035994+0000","last_clean":"2024-04-26T21:31:22.035994+0000","last_became_active":"2024-04-26T21:31:22.034795+0000","last_became_peered":"2024-04-26T21:31:22.034795+0000","last_unstale":"2024-04-26T21:31:22.035994+0000","last_undegraded":"2024-04-26T21:31:22.035994+0000","last_fullsized":"2024-04-26T21:31:22.035994+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:57.990920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843951+0000","last_change":"2024-04-26T21:30:53.824573+0000","last_active":"2024-04-26T21:31:21.843951+0000","last_peered":"2024-04-26T21:31:21.843951+0000","last_clean":"2024-04-26T21:31:21.843951+0000","last_became_active":"2024-04-26T21:30:53.824242+0000","last_became_peered":"2024-04-26T21:30:53.824242+0000","last_unstale":"2024-04-26T21:31:21.843951+0000","last_undegraded":"2024-04-26T21:31:21.843951+0000","last_fullsized":"2024-04-26T21:31:21.843951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:12.271919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843631+0000","last_change":"2024-04-26T21:30:53.827619+0000","last_active":"2024-04-26T21:31:21.843631+0000","last_peered":"2024-04-26T21:31:21.843631+0000","last_clean":"2024-04-26T21:31:21.843631+0000","last_became_active":"2024-04-26T21:30:53.827301+0000","last_became_peered":"2024-04-26T21:30:53.827301+0000","last_unstale":"2024-04-26T21:31:21.843631+0000","last_undegraded":"2024-04-26T21:31:21.843631+0000","last_fullsized":"2024-04-26T21:31:21.843631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:25.152302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.830809+0000","last_change":"2024-04-26T21:30:53.825785+0000","last_active":"2024-04-26T21:31:21.830809+0000","last_peered":"2024-04-26T21:31:21.830809+0000","last_clean":"2024-04-26T21:31:21.830809+0000","last_became_active":"2024-04-26T21:30:53.821673+0000","last_became_peered":"2024-04-26T21:30:53.821673+0000","last_unstale":"2024-04-26T21:31:21.830809+0000","last_undegraded":"2024-04-26T21:31:21.830809+0000","last_fullsized":"2024-04-26T21:31:21.830809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:39.138995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.839087+0000","last_change":"2024-04-26T21:30:53.834768+0000","last_active":"2024-04-26T21:31:21.839087+0000","last_peered":"2024-04-26T21:31:21.839087+0000","last_clean":"2024-04-26T21:31:21.839087+0000","last_became_active":"2024-04-26T21:30:53.831624+0000","last_became_peered":"2024-04-26T21:30:53.831624+0000","last_unstale":"2024-04-26T21:31:21.839087+0000","last_undegraded":"2024-04-26T21:31:21.839087+0000","last_fullsized":"2024-04-26T21:31:21.839087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:49.228987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033911+0000","last_change":"2024-04-26T21:30:53.828348+0000","last_active":"2024-04-26T21:31:22.033911+0000","last_peered":"2024-04-26T21:31:22.033911+0000","last_clean":"2024-04-26T21:31:22.033911+0000","last_became_active":"2024-04-26T21:30:53.827926+0000","last_became_peered":"2024-04-26T21:30:53.827926+0000","last_unstale":"2024-04-26T21:31:22.033911+0000","last_undegraded":"2024-04-26T21:31:22.033911+0000","last_fullsized":"2024-04-26T21:31:22.033911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.845547+0000","last_change":"2024-04-26T21:30:53.824205+0000","last_active":"2024-04-26T21:31:21.845547+0000","last_peered":"2024-04-26T21:31:21.845547+0000","last_clean":"2024-04-26T21:31:21.845547+0000","last_became_active":"2024-04-26T21:30:53.823787+0000","last_became_peered":"2024-04-26T21:30:53.823787+0000","last_unstale":"2024-04-26T21:31:21.845547+0000","last_undegraded":"2024-04-26T21:31:21.845547+0000","last_fullsized":"2024-04-26T21:31:21.845547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:23.325924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:18.918525+0000","last_change":"2024-04-26T21:30:52.803835+0000","last_active":"2024-04-26T21:32:18.918525+0000","last_peered":"2024-04-26T21:32:18.918525+0000","last_clean":"2024-04-26T21:32:18.918525+0000","last_became_active":"2024-04-26T21:30:52.803541+0000","last_became_peered":"2024-04-26T21:30:52.803541+0000","last_unstale":"2024-04-26T21:32:18.918525+0000","last_undegraded":"2024-04-26T21:32:18.918525+0000","last_fullsized":"2024-04-26T21:32:18.918525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:16.314776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842833+0000","last_change":"2024-04-26T21:30:53.823811+0000","last_active":"2024-04-26T21:31:21.842833+0000","last_peered":"2024-04-26T21:31:21.842833+0000","last_clean":"2024-04-26T21:31:21.842833+0000","last_became_active":"2024-04-26T21:30:53.823284+0000","last_became_peered":"2024-04-26T21:30:53.823284+0000","last_unstale":"2024-04-26T21:31:21.842833+0000","last_undegraded":"2024-04-26T21:31:21.842833+0000","last_fullsized":"2024-04-26T21:31:21.842833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:05.627445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843011+0000","last_change":"2024-04-26T21:30:52.810994+0000","last_active":"2024-04-26T21:31:21.843011+0000","last_peered":"2024-04-26T21:31:21.843011+0000","last_clean":"2024-04-26T21:31:21.843011+0000","last_became_active":"2024-04-26T21:30:52.809509+0000","last_became_peered":"2024-04-26T21:30:52.809509+0000","last_unstale":"2024-04-26T21:31:21.843011+0000","last_undegraded":"2024-04-26T21:31:21.843011+0000","last_fullsized":"2024-04-26T21:31:21.843011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:24.366289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844835+0000","last_change":"2024-04-26T21:30:53.830471+0000","last_active":"2024-04-26T21:31:21.844835+0000","last_peered":"2024-04-26T21:31:21.844835+0000","last_clean":"2024-04-26T21:31:21.844835+0000","last_became_active":"2024-04-26T21:30:53.829841+0000","last_became_peered":"2024-04-26T21:30:53.829841+0000","last_unstale":"2024-04-26T21:31:21.844835+0000","last_undegraded":"2024-04-26T21:31:21.844835+0000","last_fullsized":"2024-04-26T21:31:21.844835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:25.988137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.849302+0000","last_change":"2024-04-26T21:30:52.812564+0000","last_active":"2024-04-26T21:31:21.849302+0000","last_peered":"2024-04-26T21:31:21.849302+0000","last_clean":"2024-04-26T21:31:21.849302+0000","last_became_active":"2024-04-26T21:30:52.811090+0000","last_became_peered":"2024-04-26T21:30:52.811090+0000","last_unstale":"2024-04-26T21:31:21.849302+0000","last_undegraded":"2024-04-26T21:31:21.849302+0000","last_fullsized":"2024-04-26T21:31:21.849302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:51.522689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831380+0000","last_change":"2024-04-26T21:30:53.828398+0000","last_active":"2024-04-26T21:31:21.831380+0000","last_peered":"2024-04-26T21:31:21.831380+0000","last_clean":"2024-04-26T21:31:21.831380+0000","last_became_active":"2024-04-26T21:30:53.827920+0000","last_became_peered":"2024-04-26T21:30:53.827920+0000","last_unstale":"2024-04-26T21:31:21.831380+0000","last_undegraded":"2024-04-26T21:31:21.831380+0000","last_fullsized":"2024-04-26T21:31:21.831380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:06.463121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848430+0000","last_change":"2024-04-26T21:30:52.798297+0000","last_active":"2024-04-26T21:31:21.848430+0000","last_peered":"2024-04-26T21:31:21.848430+0000","last_clean":"2024-04-26T21:31:21.848430+0000","last_became_active":"2024-04-26T21:30:52.797998+0000","last_became_peered":"2024-04-26T21:30:52.797998+0000","last_unstale":"2024-04-26T21:31:21.848430+0000","last_undegraded":"2024-04-26T21:31:21.848430+0000","last_fullsized":"2024-04-26T21:31:21.848430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:40.790353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832850+0000","last_change":"2024-04-26T21:30:53.830846+0000","last_active":"2024-04-26T21:31:21.832850+0000","last_peered":"2024-04-26T21:31:21.832850+0000","last_clean":"2024-04-26T21:31:21.832850+0000","last_became_active":"2024-04-26T21:30:53.830294+0000","last_became_peered":"2024-04-26T21:30:53.830294+0000","last_unstale":"2024-04-26T21:31:21.832850+0000","last_undegraded":"2024-04-26T21:31:21.832850+0000","last_fullsized":"2024-04-26T21:31:21.832850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:50.929819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":104,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:15.932888+0000","last_change":"2024-04-26T21:30:52.807155+0000","last_active":"2024-04-26T21:32:15.932888+0000","last_peered":"2024-04-26T21:32:15.932888+0000","last_clean":"2024-04-26T21:32:15.932888+0000","last_became_active":"2024-04-26T21:30:52.805230+0000","last_became_peered":"2024-04-26T21:30:52.805230+0000","last_unstale":"2024-04-26T21:32:15.932888+0000","last_undegraded":"2024-04-26T21:32:15.932888+0000","last_fullsized":"2024-04-26T21:32:15.932888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:44.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848460+0000","last_change":"2024-04-26T21:30:53.810334+0000","last_active":"2024-04-26T21:31:21.848460+0000","last_peered":"2024-04-26T21:31:21.848460+0000","last_clean":"2024-04-26T21:31:21.848460+0000","last_became_active":"2024-04-26T21:30:53.809867+0000","last_became_peered":"2024-04-26T21:30:53.809867+0000","last_unstale":"2024-04-26T21:31:21.848460+0000","last_undegraded":"2024-04-26T21:31:21.848460+0000","last_fullsized":"2024-04-26T21:31:21.848460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:46.377262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.947127+0000","last_change":"2024-04-26T21:30:52.804775+0000","last_active":"2024-04-26T21:31:21.947127+0000","last_peered":"2024-04-26T21:31:21.947127+0000","last_clean":"2024-04-26T21:31:21.947127+0000","last_became_active":"2024-04-26T21:30:52.804319+0000","last_became_peered":"2024-04-26T21:30:52.804319+0000","last_unstale":"2024-04-26T21:31:21.947127+0000","last_undegraded":"2024-04-26T21:31:21.947127+0000","last_fullsized":"2024-04-26T21:31:21.947127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:12.761080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832486+0000","last_change":"2024-04-26T21:30:53.823021+0000","last_active":"2024-04-26T21:31:21.832486+0000","last_peered":"2024-04-26T21:31:21.832486+0000","last_clean":"2024-04-26T21:31:21.832486+0000","last_became_active":"2024-04-26T21:30:53.822633+0000","last_became_peered":"2024-04-26T21:30:53.822633+0000","last_unstale":"2024-04-26T21:31:21.832486+0000","last_undegraded":"2024-04-26T21:31:21.832486+0000","last_fullsized":"2024-04-26T21:31:21.832486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:32.178472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827806+0000","last_change":"2024-04-26T21:30:52.796729+0000","last_active":"2024-04-26T21:31:21.827806+0000","last_peered":"2024-04-26T21:31:21.827806+0000","last_clean":"2024-04-26T21:31:21.827806+0000","last_became_active":"2024-04-26T21:30:52.796341+0000","last_became_peered":"2024-04-26T21:30:52.796341+0000","last_unstale":"2024-04-26T21:31:21.827806+0000","last_undegraded":"2024-04-26T21:31:21.827806+0000","last_fullsized":"2024-04-26T21:31:21.827806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:30.434541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.841762+0000","last_change":"2024-04-26T21:30:53.816136+0000","last_active":"2024-04-26T21:31:21.841762+0000","last_peered":"2024-04-26T21:31:21.841762+0000","last_clean":"2024-04-26T21:31:21.841762+0000","last_became_active":"2024-04-26T21:30:53.815758+0000","last_became_peered":"2024-04-26T21:30:53.815758+0000","last_unstale":"2024-04-26T21:31:21.841762+0000","last_undegraded":"2024-04-26T21:31:21.841762+0000","last_fullsized":"2024-04-26T21:31:21.841762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:41.606483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831981+0000","last_change":"2024-04-26T21:30:52.804102+0000","last_active":"2024-04-26T21:31:21.831981+0000","last_peered":"2024-04-26T21:31:21.831981+0000","last_clean":"2024-04-26T21:31:21.831981+0000","last_became_active":"2024-04-26T21:30:52.803808+0000","last_became_peered":"2024-04-26T21:30:52.803808+0000","last_unstale":"2024-04-26T21:31:21.831981+0000","last_undegraded":"2024-04-26T21:31:21.831981+0000","last_fullsized":"2024-04-26T21:31:21.831981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:16.357651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033354+0000","last_change":"2024-04-26T21:30:53.822475+0000","last_active":"2024-04-26T21:31:22.033354+0000","last_peered":"2024-04-26T21:31:22.033354+0000","last_clean":"2024-04-26T21:31:22.033354+0000","last_became_active":"2024-04-26T21:30:53.822077+0000","last_became_peered":"2024-04-26T21:30:53.822077+0000","last_unstale":"2024-04-26T21:31:22.033354+0000","last_undegraded":"2024-04-26T21:31:22.033354+0000","last_fullsized":"2024-04-26T21:31:22.033354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:09.289557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:15.931076+0000","last_change":"2024-04-26T21:30:52.802080+0000","last_active":"2024-04-26T21:32:15.931076+0000","last_peered":"2024-04-26T21:32:15.931076+0000","last_clean":"2024-04-26T21:32:15.931076+0000","last_became_active":"2024-04-26T21:30:52.801645+0000","last_became_peered":"2024-04-26T21:30:52.801645+0000","last_unstale":"2024-04-26T21:32:15.931076+0000","last_undegraded":"2024-04-26T21:32:15.931076+0000","last_fullsized":"2024-04-26T21:32:15.931076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:00.213924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832072+0000","last_change":"2024-04-26T21:30:53.832936+0000","last_active":"2024-04-26T21:31:21.832072+0000","last_peered":"2024-04-26T21:31:21.832072+0000","last_clean":"2024-04-26T21:31:21.832072+0000","last_became_active":"2024-04-26T21:30:53.830821+0000","last_became_peered":"2024-04-26T21:30:53.830821+0000","last_unstale":"2024-04-26T21:31:21.832072+0000","last_undegraded":"2024-04-26T21:31:21.832072+0000","last_fullsized":"2024-04-26T21:31:21.832072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:08.540653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520227+0000","last_change":"2024-04-26T21:30:52.812687+0000","last_active":"2024-04-26T21:31:22.520227+0000","last_peered":"2024-04-26T21:31:22.520227+0000","last_clean":"2024-04-26T21:31:22.520227+0000","last_became_active":"2024-04-26T21:30:52.812055+0000","last_became_peered":"2024-04-26T21:30:52.812055+0000","last_unstale":"2024-04-26T21:31:22.520227+0000","last_undegraded":"2024-04-26T21:31:22.520227+0000","last_fullsized":"2024-04-26T21:31:22.520227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:51.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844669+0000","last_change":"2024-04-26T21:30:53.829887+0000","last_active":"2024-04-26T21:31:21.844669+0000","last_peered":"2024-04-26T21:31:21.844669+0000","last_clean":"2024-04-26T21:31:21.844669+0000","last_became_active":"2024-04-26T21:30:53.829355+0000","last_became_peered":"2024-04-26T21:30:53.829355+0000","last_unstale":"2024-04-26T21:31:21.844669+0000","last_undegraded":"2024-04-26T21:31:21.844669+0000","last_fullsized":"2024-04-26T21:31:21.844669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:49.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.833113+0000","last_change":"2024-04-26T21:30:52.808859+0000","last_active":"2024-04-26T21:31:21.833113+0000","last_peered":"2024-04-26T21:31:21.833113+0000","last_clean":"2024-04-26T21:31:21.833113+0000","last_became_active":"2024-04-26T21:30:52.807232+0000","last_became_peered":"2024-04-26T21:30:52.807232+0000","last_unstale":"2024-04-26T21:31:21.833113+0000","last_undegraded":"2024-04-26T21:31:21.833113+0000","last_fullsized":"2024-04-26T21:31:21.833113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:44.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831743+0000","last_change":"2024-04-26T21:30:53.829723+0000","last_active":"2024-04-26T21:31:21.831743+0000","last_peered":"2024-04-26T21:31:21.831743+0000","last_clean":"2024-04-26T21:31:21.831743+0000","last_became_active":"2024-04-26T21:30:53.827799+0000","last_became_peered":"2024-04-26T21:30:53.827799+0000","last_unstale":"2024-04-26T21:31:21.831743+0000","last_undegraded":"2024-04-26T21:31:21.831743+0000","last_fullsized":"2024-04-26T21:31:21.831743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:20.798100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":205,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:15.939690+0000","last_change":"2024-04-26T21:30:52.812217+0000","last_active":"2024-04-26T21:32:15.939690+0000","last_peered":"2024-04-26T21:32:15.939690+0000","last_clean":"2024-04-26T21:32:15.939690+0000","last_became_active":"2024-04-26T21:30:52.811730+0000","last_became_peered":"2024-04-26T21:30:52.811730+0000","last_unstale":"2024-04-26T21:32:15.939690+0000","last_undegraded":"2024-04-26T21:32:15.939690+0000","last_fullsized":"2024-04-26T21:32:15.939690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:08.012161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52244,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":130,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848793+0000","last_change":"2024-04-26T21:30:53.813388+0000","last_active":"2024-04-26T21:31:21.848793+0000","last_peered":"2024-04-26T21:31:21.848793+0000","last_clean":"2024-04-26T21:31:21.848793+0000","last_became_active":"2024-04-26T21:30:53.812972+0000","last_became_peered":"2024-04-26T21:30:53.812972+0000","last_unstale":"2024-04-26T21:31:21.848793+0000","last_undegraded":"2024-04-26T21:31:21.848793+0000","last_fullsized":"2024-04-26T21:31:21.848793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:22.480641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844911+0000","last_change":"2024-04-26T21:30:52.804426+0000","last_active":"2024-04-26T21:31:21.844911+0000","last_peered":"2024-04-26T21:31:21.844911+0000","last_clean":"2024-04-26T21:31:21.844911+0000","last_became_active":"2024-04-26T21:30:52.804083+0000","last_became_peered":"2024-04-26T21:30:52.804083+0000","last_unstale":"2024-04-26T21:31:21.844911+0000","last_undegraded":"2024-04-26T21:31:21.844911+0000","last_fullsized":"2024-04-26T21:31:21.844911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:10.164976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827875+0000","last_change":"2024-04-26T21:30:53.829326+0000","last_active":"2024-04-26T21:31:21.827875+0000","last_peered":"2024-04-26T21:31:21.827875+0000","last_clean":"2024-04-26T21:31:21.827875+0000","last_became_active":"2024-04-26T21:30:53.828863+0000","last_became_peered":"2024-04-26T21:30:53.828863+0000","last_unstale":"2024-04-26T21:31:21.827875+0000","last_undegraded":"2024-04-26T21:31:21.827875+0000","last_fullsized":"2024-04-26T21:31:21.827875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:56.756187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831957+0000","last_change":"2024-04-26T21:30:52.805302+0000","last_active":"2024-04-26T21:31:21.831957+0000","last_peered":"2024-04-26T21:31:21.831957+0000","last_clean":"2024-04-26T21:31:21.831957+0000","last_became_active":"2024-04-26T21:30:52.804869+0000","last_became_peered":"2024-04-26T21:30:52.804869+0000","last_unstale":"2024-04-26T21:31:21.831957+0000","last_undegraded":"2024-04-26T21:31:21.831957+0000","last_fullsized":"2024-04-26T21:31:21.831957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:21.299474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519967+0000","last_change":"2024-04-26T21:30:53.823827+0000","last_active":"2024-04-26T21:31:22.519967+0000","last_peered":"2024-04-26T21:31:22.519967+0000","last_clean":"2024-04-26T21:31:22.519967+0000","last_became_active":"2024-04-26T21:30:53.823492+0000","last_became_peered":"2024-04-26T21:30:53.823492+0000","last_unstale":"2024-04-26T21:31:22.519967+0000","last_undegraded":"2024-04-26T21:31:22.519967+0000","last_fullsized":"2024-04-26T21:31:22.519967+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.514753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":106,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:17.942971+0000","last_change":"2024-04-26T21:30:52.801468+0000","last_active":"2024-04-26T21:32:17.942971+0000","last_peered":"2024-04-26T21:32:17.942971+0000","last_clean":"2024-04-26T21:32:17.942971+0000","last_became_active":"2024-04-26T21:30:52.801071+0000","last_became_peered":"2024-04-26T21:30:52.801071+0000","last_unstale":"2024-04-26T21:32:17.942971+0000","last_undegraded":"2024-04-26T21:32:17.942971+0000","last_fullsized":"2024-04-26T21:32:17.942971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:52.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519357+0000","last_change":"2024-04-26T21:30:53.824075+0000","last_active":"2024-04-26T21:31:22.519357+0000","last_peered":"2024-04-26T21:31:22.519357+0000","last_clean":"2024-04-26T21:31:22.519357+0000","last_became_active":"2024-04-26T21:30:53.823753+0000","last_became_peered":"2024-04-26T21:30:53.823753+0000","last_unstale":"2024-04-26T21:31:22.519357+0000","last_undegraded":"2024-04-26T21:31:22.519357+0000","last_fullsized":"2024-04-26T21:31:22.519357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:44:09.294481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.154980+0000","last_change":"2024-04-26T21:31:22.153880+0000","last_active":"2024-04-26T21:31:22.154980+0000","last_peered":"2024-04-26T21:31:22.154980+0000","last_clean":"2024-04-26T21:31:22.154980+0000","last_became_active":"2024-04-26T21:31:22.085354+0000","last_became_peered":"2024-04-26T21:31:22.085354+0000","last_unstale":"2024-04-26T21:31:22.154980+0000","last_undegraded":"2024-04-26T21:31:22.154980+0000","last_fullsized":"2024-04-26T21:31:22.154980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:05.687450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831958+0000","last_change":"2024-04-26T21:30:53.835273+0000","last_active":"2024-04-26T21:31:21.831958+0000","last_peered":"2024-04-26T21:31:21.831958+0000","last_clean":"2024-04-26T21:31:21.831958+0000","last_became_active":"2024-04-26T21:30:53.834922+0000","last_became_peered":"2024-04-26T21:30:53.834922+0000","last_unstale":"2024-04-26T21:31:21.831958+0000","last_undegraded":"2024-04-26T21:31:21.831958+0000","last_fullsized":"2024-04-26T21:31:21.831958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:43.992579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848829+0000","last_change":"2024-04-26T21:30:52.797639+0000","last_active":"2024-04-26T21:31:21.848829+0000","last_peered":"2024-04-26T21:31:21.848829+0000","last_clean":"2024-04-26T21:31:21.848829+0000","last_became_active":"2024-04-26T21:30:52.796576+0000","last_became_peered":"2024-04-26T21:30:52.796576+0000","last_unstale":"2024-04-26T21:31:21.848829+0000","last_undegraded":"2024-04-26T21:31:21.848829+0000","last_fullsized":"2024-04-26T21:31:21.848829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.730232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.838255+0000","last_change":"2024-04-26T21:30:53.836190+0000","last_active":"2024-04-26T21:31:21.838255+0000","last_peered":"2024-04-26T21:31:21.838255+0000","last_clean":"2024-04-26T21:31:21.838255+0000","last_became_active":"2024-04-26T21:30:53.835302+0000","last_became_peered":"2024-04-26T21:30:53.835302+0000","last_unstale":"2024-04-26T21:31:21.838255+0000","last_undegraded":"2024-04-26T21:31:21.838255+0000","last_fullsized":"2024-04-26T21:31:21.838255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:13.222578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033576+0000","last_change":"2024-04-26T21:30:52.809928+0000","last_active":"2024-04-26T21:31:22.033576+0000","last_peered":"2024-04-26T21:31:22.033576+0000","last_clean":"2024-04-26T21:31:22.033576+0000","last_became_active":"2024-04-26T21:30:52.808667+0000","last_became_peered":"2024-04-26T21:30:52.808667+0000","last_unstale":"2024-04-26T21:31:22.033576+0000","last_undegraded":"2024-04-26T21:31:22.033576+0000","last_fullsized":"2024-04-26T21:31:22.033576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:15.375105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948044+0000","last_change":"2024-04-26T21:30:53.836649+0000","last_active":"2024-04-26T21:31:21.948044+0000","last_peered":"2024-04-26T21:31:21.948044+0000","last_clean":"2024-04-26T21:31:21.948044+0000","last_became_active":"2024-04-26T21:30:53.836252+0000","last_became_peered":"2024-04-26T21:30:53.836252+0000","last_unstale":"2024-04-26T21:31:21.948044+0000","last_undegraded":"2024-04-26T21:31:21.948044+0000","last_fullsized":"2024-04-26T21:31:21.948044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:53.797872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":105,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:18.919570+0000","last_change":"2024-04-26T21:30:52.808196+0000","last_active":"2024-04-26T21:32:18.919570+0000","last_peered":"2024-04-26T21:32:18.919570+0000","last_clean":"2024-04-26T21:32:18.919570+0000","last_became_active":"2024-04-26T21:30:52.806445+0000","last_became_peered":"2024-04-26T21:30:52.806445+0000","last_unstale":"2024-04-26T21:32:18.919570+0000","last_undegraded":"2024-04-26T21:32:18.919570+0000","last_fullsized":"2024-04-26T21:32:18.919570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:39.815485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948196+0000","last_change":"2024-04-26T21:30:53.837998+0000","last_active":"2024-04-26T21:31:21.948196+0000","last_peered":"2024-04-26T21:31:21.948196+0000","last_clean":"2024-04-26T21:31:21.948196+0000","last_became_active":"2024-04-26T21:30:53.837443+0000","last_became_peered":"2024-04-26T21:30:53.837443+0000","last_unstale":"2024-04-26T21:31:21.948196+0000","last_undegraded":"2024-04-26T21:31:21.948196+0000","last_fullsized":"2024-04-26T21:31:21.948196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:04.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.833579+0000","last_change":"2024-04-26T21:30:53.835501+0000","last_active":"2024-04-26T21:31:21.833579+0000","last_peered":"2024-04-26T21:31:21.833579+0000","last_clean":"2024-04-26T21:31:21.833579+0000","last_became_active":"2024-04-26T21:30:53.835096+0000","last_became_peered":"2024-04-26T21:30:53.835096+0000","last_unstale":"2024-04-26T21:31:21.833579+0000","last_undegraded":"2024-04-26T21:31:21.833579+0000","last_fullsized":"2024-04-26T21:31:21.833579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:26.178001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:16.938861+0000","last_change":"2024-04-26T21:30:52.805357+0000","last_active":"2024-04-26T21:32:16.938861+0000","last_peered":"2024-04-26T21:32:16.938861+0000","last_clean":"2024-04-26T21:32:16.938861+0000","last_became_active":"2024-04-26T21:30:52.804902+0000","last_became_peered":"2024-04-26T21:30:52.804902+0000","last_unstale":"2024-04-26T21:32:16.938861+0000","last_undegraded":"2024-04-26T21:32:16.938861+0000","last_fullsized":"2024-04-26T21:32:16.938861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:29.276141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842272+0000","last_change":"2024-04-26T21:30:53.835448+0000","last_active":"2024-04-26T21:31:21.842272+0000","last_peered":"2024-04-26T21:31:21.842272+0000","last_clean":"2024-04-26T21:31:21.842272+0000","last_became_active":"2024-04-26T21:30:53.835054+0000","last_became_peered":"2024-04-26T21:30:53.835054+0000","last_unstale":"2024-04-26T21:31:21.842272+0000","last_undegraded":"2024-04-26T21:31:21.842272+0000","last_fullsized":"2024-04-26T21:31:21.842272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:37.892713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.081775+0000","last_change":"2024-04-26T21:30:52.792942+0000","last_active":"2024-04-26T21:31:22.081775+0000","last_peered":"2024-04-26T21:31:22.081775+0000","last_clean":"2024-04-26T21:31:22.081775+0000","last_became_active":"2024-04-26T21:30:52.792553+0000","last_became_peered":"2024-04-26T21:30:52.792553+0000","last_unstale":"2024-04-26T21:31:22.081775+0000","last_undegraded":"2024-04-26T21:31:22.081775+0000","last_fullsized":"2024-04-26T21:31:22.081775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.915883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519688+0000","last_change":"2024-04-26T21:29:15.217076+0000","last_active":"2024-04-26T21:31:22.519688+0000","last_peered":"2024-04-26T21:31:22.519688+0000","last_clean":"2024-04-26T21:31:22.519688+0000","last_became_active":"2024-04-26T21:29:15.215786+0000","last_became_peered":"2024-04-26T21:29:15.215786+0000","last_unstale":"2024-04-26T21:31:22.519688+0000","last_undegraded":"2024-04-26T21:31:22.519688+0000","last_fullsized":"2024-04-26T21:31:22.519688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_clean_scrub_stamp":"2024-04-26T21:24:37.649628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:47.795571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827632+0000","last_change":"2024-04-26T21:30:53.835344+0000","last_active":"2024-04-26T21:31:21.827632+0000","last_peered":"2024-04-26T21:31:21.827632+0000","last_clean":"2024-04-26T21:31:21.827632+0000","last_became_active":"2024-04-26T21:30:53.834838+0000","last_became_peered":"2024-04-26T21:30:53.834838+0000","last_unstale":"2024-04-26T21:31:21.827632+0000","last_undegraded":"2024-04-26T21:31:21.827632+0000","last_fullsized":"2024-04-26T21:31:21.827632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:37.588955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519851+0000","last_change":"2024-04-26T21:30:52.808173+0000","last_active":"2024-04-26T21:31:22.519851+0000","last_peered":"2024-04-26T21:31:22.519851+0000","last_clean":"2024-04-26T21:31:22.519851+0000","last_became_active":"2024-04-26T21:30:52.807688+0000","last_became_peered":"2024-04-26T21:30:52.807688+0000","last_unstale":"2024-04-26T21:31:22.519851+0000","last_undegraded":"2024-04-26T21:31:22.519851+0000","last_fullsized":"2024-04-26T21:31:22.519851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:58.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848703+0000","last_change":"2024-04-26T21:30:52.810654+0000","last_active":"2024-04-26T21:31:21.848703+0000","last_peered":"2024-04-26T21:31:21.848703+0000","last_clean":"2024-04-26T21:31:21.848703+0000","last_became_active":"2024-04-26T21:30:52.808959+0000","last_became_peered":"2024-04-26T21:30:52.808959+0000","last_unstale":"2024-04-26T21:31:21.848703+0000","last_undegraded":"2024-04-26T21:31:21.848703+0000","last_fullsized":"2024-04-26T21:31:21.848703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:59.134325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827988+0000","last_change":"2024-04-26T21:30:53.816961+0000","last_active":"2024-04-26T21:31:21.827988+0000","last_peered":"2024-04-26T21:31:21.827988+0000","last_clean":"2024-04-26T21:31:21.827988+0000","last_became_active":"2024-04-26T21:30:53.816571+0000","last_became_peered":"2024-04-26T21:30:53.816571+0000","last_unstale":"2024-04-26T21:31:21.827988+0000","last_undegraded":"2024-04-26T21:31:21.827988+0000","last_fullsized":"2024-04-26T21:31:21.827988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:43.218823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033616+0000","last_change":"2024-04-26T21:30:52.801027+0000","last_active":"2024-04-26T21:31:22.033616+0000","last_peered":"2024-04-26T21:31:22.033616+0000","last_clean":"2024-04-26T21:31:22.033616+0000","last_became_active":"2024-04-26T21:30:52.797107+0000","last_became_peered":"2024-04-26T21:30:52.797107+0000","last_unstale":"2024-04-26T21:31:22.033616+0000","last_undegraded":"2024-04-26T21:31:22.033616+0000","last_fullsized":"2024-04-26T21:31:22.033616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:23.739673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842740+0000","last_change":"2024-04-26T21:30:53.821946+0000","last_active":"2024-04-26T21:31:21.842740+0000","last_peered":"2024-04-26T21:31:21.842740+0000","last_clean":"2024-04-26T21:31:21.842740+0000","last_became_active":"2024-04-26T21:30:53.821433+0000","last_became_peered":"2024-04-26T21:30:53.821433+0000","last_unstale":"2024-04-26T21:31:21.842740+0000","last_undegraded":"2024-04-26T21:31:21.842740+0000","last_fullsized":"2024-04-26T21:31:21.842740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:30.426334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.082756+0000","last_change":"2024-04-26T21:30:52.806302+0000","last_active":"2024-04-26T21:31:22.082756+0000","last_peered":"2024-04-26T21:31:22.082756+0000","last_clean":"2024-04-26T21:31:22.082756+0000","last_became_active":"2024-04-26T21:30:52.803402+0000","last_became_peered":"2024-04-26T21:30:52.803402+0000","last_unstale":"2024-04-26T21:31:22.082756+0000","last_undegraded":"2024-04-26T21:31:22.082756+0000","last_fullsized":"2024-04-26T21:31:22.082756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:07.163372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844868+0000","last_change":"2024-04-26T21:30:53.833803+0000","last_active":"2024-04-26T21:31:21.844868+0000","last_peered":"2024-04-26T21:31:21.844868+0000","last_clean":"2024-04-26T21:31:21.844868+0000","last_became_active":"2024-04-26T21:30:53.833378+0000","last_became_peered":"2024-04-26T21:30:53.833378+0000","last_unstale":"2024-04-26T21:31:21.844868+0000","last_undegraded":"2024-04-26T21:31:21.844868+0000","last_fullsized":"2024-04-26T21:31:21.844868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:58.505849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.830812+0000","last_change":"2024-04-26T21:30:52.810031+0000","last_active":"2024-04-26T21:31:21.830812+0000","last_peered":"2024-04-26T21:31:21.830812+0000","last_clean":"2024-04-26T21:31:21.830812+0000","last_became_active":"2024-04-26T21:30:52.809743+0000","last_became_peered":"2024-04-26T21:30:52.809743+0000","last_unstale":"2024-04-26T21:31:21.830812+0000","last_undegraded":"2024-04-26T21:31:21.830812+0000","last_fullsized":"2024-04-26T21:31:21.830812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:05.723934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.845313+0000","last_change":"2024-04-26T21:30:53.822563+0000","last_active":"2024-04-26T21:31:21.845313+0000","last_peered":"2024-04-26T21:31:21.845313+0000","last_clean":"2024-04-26T21:31:21.845313+0000","last_became_active":"2024-04-26T21:30:53.821848+0000","last_became_peered":"2024-04-26T21:30:53.821848+0000","last_unstale":"2024-04-26T21:31:21.845313+0000","last_undegraded":"2024-04-26T21:31:21.845313+0000","last_fullsized":"2024-04-26T21:31:21.845313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:39.861976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842264+0000","last_change":"2024-04-26T21:30:52.792458+0000","last_active":"2024-04-26T21:31:21.842264+0000","last_peered":"2024-04-26T21:31:21.842264+0000","last_clean":"2024-04-26T21:31:21.842264+0000","last_became_active":"2024-04-26T21:30:52.792087+0000","last_became_peered":"2024-04-26T21:30:52.792087+0000","last_unstale":"2024-04-26T21:31:21.842264+0000","last_undegraded":"2024-04-26T21:31:21.842264+0000","last_fullsized":"2024-04-26T21:31:21.842264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:46.420509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837340+0000","last_change":"2024-04-26T21:30:53.826028+0000","last_active":"2024-04-26T21:31:21.837340+0000","last_peered":"2024-04-26T21:31:21.837340+0000","last_clean":"2024-04-26T21:31:21.837340+0000","last_became_active":"2024-04-26T21:30:53.822405+0000","last_became_peered":"2024-04-26T21:30:53.822405+0000","last_unstale":"2024-04-26T21:31:21.837340+0000","last_undegraded":"2024-04-26T21:31:21.837340+0000","last_fullsized":"2024-04-26T21:31:21.837340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:47.501074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.862987+0000","last_change":"2024-04-26T21:31:21.861728+0000","last_active":"2024-04-26T21:31:21.862987+0000","last_peered":"2024-04-26T21:31:21.862987+0000","last_clean":"2024-04-26T21:31:21.862987+0000","last_became_active":"2024-04-26T21:31:21.850330+0000","last_became_peered":"2024-04-26T21:31:21.850330+0000","last_unstale":"2024-04-26T21:31:21.862987+0000","last_undegraded":"2024-04-26T21:31:21.862987+0000","last_fullsized":"2024-04-26T21:31:21.862987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:05.532071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842726+0000","last_change":"2024-04-26T21:30:53.826016+0000","last_active":"2024-04-26T21:31:21.842726+0000","last_peered":"2024-04-26T21:31:21.842726+0000","last_clean":"2024-04-26T21:31:21.842726+0000","last_became_active":"2024-04-26T21:30:53.824050+0000","last_became_peered":"2024-04-26T21:30:53.824050+0000","last_unstale":"2024-04-26T21:31:21.842726+0000","last_undegraded":"2024-04-26T21:31:21.842726+0000","last_fullsized":"2024-04-26T21:31:21.842726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:35.816936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.847822+0000","last_change":"2024-04-26T21:30:52.811142+0000","last_active":"2024-04-26T21:31:21.847822+0000","last_peered":"2024-04-26T21:31:21.847822+0000","last_clean":"2024-04-26T21:31:21.847822+0000","last_became_active":"2024-04-26T21:30:52.810732+0000","last_became_peered":"2024-04-26T21:30:52.810732+0000","last_unstale":"2024-04-26T21:31:21.847822+0000","last_undegraded":"2024-04-26T21:31:21.847822+0000","last_fullsized":"2024-04-26T21:31:21.847822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:21.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519829+0000","last_change":"2024-04-26T21:30:53.832000+0000","last_active":"2024-04-26T21:31:22.519829+0000","last_peered":"2024-04-26T21:31:22.519829+0000","last_clean":"2024-04-26T21:31:22.519829+0000","last_became_active":"2024-04-26T21:30:53.831619+0000","last_became_peered":"2024-04-26T21:30:53.831619+0000","last_unstale":"2024-04-26T21:31:22.519829+0000","last_undegraded":"2024-04-26T21:31:22.519829+0000","last_fullsized":"2024-04-26T21:31:22.519829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:11.656424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:14.929434+0000","last_change":"2024-04-26T21:30:52.810311+0000","last_active":"2024-04-26T21:32:14.929434+0000","last_peered":"2024-04-26T21:32:14.929434+0000","last_clean":"2024-04-26T21:32:14.929434+0000","last_became_active":"2024-04-26T21:30:52.809976+0000","last_became_peered":"2024-04-26T21:30:52.809976+0000","last_unstale":"2024-04-26T21:32:14.929434+0000","last_undegraded":"2024-04-26T21:32:14.929434+0000","last_fullsized":"2024-04-26T21:32:14.929434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:49.146730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520562+0000","last_change":"2024-04-26T21:30:53.830315+0000","last_active":"2024-04-26T21:31:22.520562+0000","last_peered":"2024-04-26T21:31:22.520562+0000","last_clean":"2024-04-26T21:31:22.520562+0000","last_became_active":"2024-04-26T21:30:53.829439+0000","last_became_peered":"2024-04-26T21:30:53.829439+0000","last_unstale":"2024-04-26T21:31:22.520562+0000","last_undegraded":"2024-04-26T21:31:22.520562+0000","last_fullsized":"2024-04-26T21:31:22.520562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:49.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832532+0000","last_change":"2024-04-26T21:30:52.802392+0000","last_active":"2024-04-26T21:31:21.832532+0000","last_peered":"2024-04-26T21:31:21.832532+0000","last_clean":"2024-04-26T21:31:21.832532+0000","last_became_active":"2024-04-26T21:30:52.799978+0000","last_became_peered":"2024-04-26T21:30:52.799978+0000","last_unstale":"2024-04-26T21:31:21.832532+0000","last_undegraded":"2024-04-26T21:31:21.832532+0000","last_fullsized":"2024-04-26T21:31:21.832532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:45.993473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520063+0000","last_change":"2024-04-26T21:30:53.818118+0000","last_active":"2024-04-26T21:31:22.520063+0000","last_peered":"2024-04-26T21:31:22.520063+0000","last_clean":"2024-04-26T21:31:22.520063+0000","last_became_active":"2024-04-26T21:30:53.817712+0000","last_became_peered":"2024-04-26T21:30:53.817712+0000","last_unstale":"2024-04-26T21:31:22.520063+0000","last_undegraded":"2024-04-26T21:31:22.520063+0000","last_fullsized":"2024-04-26T21:31:22.520063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.819569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.828161+0000","last_change":"2024-04-26T21:30:52.807308+0000","last_active":"2024-04-26T21:31:21.828161+0000","last_peered":"2024-04-26T21:31:21.828161+0000","last_clean":"2024-04-26T21:31:21.828161+0000","last_became_active":"2024-04-26T21:30:52.806935+0000","last_became_peered":"2024-04-26T21:30:52.806935+0000","last_unstale":"2024-04-26T21:31:21.828161+0000","last_undegraded":"2024-04-26T21:31:21.828161+0000","last_fullsized":"2024-04-26T21:31:21.828161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:42.549564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520636+0000","last_change":"2024-04-26T21:30:53.812940+0000","last_active":"2024-04-26T21:31:22.520636+0000","last_peered":"2024-04-26T21:31:22.520636+0000","last_clean":"2024-04-26T21:31:22.520636+0000","last_became_active":"2024-04-26T21:30:53.812492+0000","last_became_peered":"2024-04-26T21:30:53.812492+0000","last_unstale":"2024-04-26T21:31:22.520636+0000","last_undegraded":"2024-04-26T21:31:22.520636+0000","last_fullsized":"2024-04-26T21:31:22.520636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:07.531226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837769+0000","last_change":"2024-04-26T21:30:53.822943+0000","last_active":"2024-04-26T21:31:21.837769+0000","last_peered":"2024-04-26T21:31:21.837769+0000","last_clean":"2024-04-26T21:31:21.837769+0000","last_became_active":"2024-04-26T21:30:53.822443+0000","last_became_peered":"2024-04-26T21:30:53.822443+0000","last_unstale":"2024-04-26T21:31:21.837769+0000","last_undegraded":"2024-04-26T21:31:21.837769+0000","last_fullsized":"2024-04-26T21:31:21.837769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:57.827470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844894+0000","last_change":"2024-04-26T21:30:53.832417+0000","last_active":"2024-04-26T21:31:21.844894+0000","last_peered":"2024-04-26T21:31:21.844894+0000","last_clean":"2024-04-26T21:31:21.844894+0000","last_became_active":"2024-04-26T21:30:53.832001+0000","last_became_peered":"2024-04-26T21:30:53.832001+0000","last_unstale":"2024-04-26T21:31:21.844894+0000","last_undegraded":"2024-04-26T21:31:21.844894+0000","last_fullsized":"2024-04-26T21:31:21.844894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:02.471766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844628+0000","last_change":"2024-04-26T21:30:53.817472+0000","last_active":"2024-04-26T21:31:21.844628+0000","last_peered":"2024-04-26T21:31:21.844628+0000","last_clean":"2024-04-26T21:31:21.844628+0000","last_became_active":"2024-04-26T21:30:53.817053+0000","last_became_peered":"2024-04-26T21:30:53.817053+0000","last_unstale":"2024-04-26T21:31:21.844628+0000","last_undegraded":"2024-04-26T21:31:21.844628+0000","last_fullsized":"2024-04-26T21:31:21.844628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:19.219132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844506+0000","last_change":"2024-04-26T21:30:53.826748+0000","last_active":"2024-04-26T21:31:21.844506+0000","last_peered":"2024-04-26T21:31:21.844506+0000","last_clean":"2024-04-26T21:31:21.844506+0000","last_became_active":"2024-04-26T21:30:53.826040+0000","last_became_peered":"2024-04-26T21:30:53.826040+0000","last_unstale":"2024-04-26T21:31:21.844506+0000","last_undegraded":"2024-04-26T21:31:21.844506+0000","last_fullsized":"2024-04-26T21:31:21.844506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:18.222414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.037153+0000","last_change":"2024-04-26T21:31:22.037153+0000","last_active":"2024-04-26T21:31:22.037153+0000","last_peered":"2024-04-26T21:31:22.037153+0000","last_clean":"2024-04-26T21:31:22.037153+0000","last_became_active":"2024-04-26T21:31:22.035984+0000","last_became_peered":"2024-04-26T21:31:22.035984+0000","last_unstale":"2024-04-26T21:31:22.037153+0000","last_undegraded":"2024-04-26T21:31:22.037153+0000","last_fullsized":"2024-04-26T21:31:22.037153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:30.880953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837888+0000","last_change":"2024-04-26T21:30:53.825396+0000","last_active":"2024-04-26T21:31:21.837888+0000","last_peered":"2024-04-26T21:31:21.837888+0000","last_clean":"2024-04-26T21:31:21.837888+0000","last_became_active":"2024-04-26T21:30:53.825042+0000","last_became_peered":"2024-04-26T21:30:53.825042+0000","last_unstale":"2024-04-26T21:31:21.837888+0000","last_undegraded":"2024-04-26T21:31:21.837888+0000","last_fullsized":"2024-04-26T21:31:21.837888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:20.064265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.846648+0000","last_change":"2024-04-26T21:30:53.827819+0000","last_active":"2024-04-26T21:31:21.846648+0000","last_peered":"2024-04-26T21:31:21.846648+0000","last_clean":"2024-04-26T21:31:21.846648+0000","last_became_active":"2024-04-26T21:30:53.827027+0000","last_became_peered":"2024-04-26T21:30:53.827027+0000","last_unstale":"2024-04-26T21:31:21.846648+0000","last_undegraded":"2024-04-26T21:31:21.846648+0000","last_fullsized":"2024-04-26T21:31:21.846648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:49.586652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842726+0000","last_change":"2024-04-26T21:30:53.824693+0000","last_active":"2024-04-26T21:31:21.842726+0000","last_peered":"2024-04-26T21:31:21.842726+0000","last_clean":"2024-04-26T21:31:21.842726+0000","last_became_active":"2024-04-26T21:30:53.824278+0000","last_became_peered":"2024-04-26T21:30:53.824278+0000","last_unstale":"2024-04-26T21:31:21.842726+0000","last_undegraded":"2024-04-26T21:31:21.842726+0000","last_fullsized":"2024-04-26T21:31:21.842726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:42.799139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831974+0000","last_change":"2024-04-26T21:30:53.823886+0000","last_active":"2024-04-26T21:31:21.831974+0000","last_peered":"2024-04-26T21:31:21.831974+0000","last_clean":"2024-04-26T21:31:21.831974+0000","last_became_active":"2024-04-26T21:30:53.823443+0000","last_became_peered":"2024-04-26T21:30:53.823443+0000","last_unstale":"2024-04-26T21:31:21.831974+0000","last_undegraded":"2024-04-26T21:31:21.831974+0000","last_fullsized":"2024-04-26T21:31:21.831974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.022073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842355+0000","last_change":"2024-04-26T21:30:53.825329+0000","last_active":"2024-04-26T21:31:21.842355+0000","last_peered":"2024-04-26T21:31:21.842355+0000","last_clean":"2024-04-26T21:31:21.842355+0000","last_became_active":"2024-04-26T21:30:53.822833+0000","last_became_peered":"2024-04-26T21:30:53.822833+0000","last_unstale":"2024-04-26T21:31:21.842355+0000","last_undegraded":"2024-04-26T21:31:21.842355+0000","last_fullsized":"2024-04-26T21:31:21.842355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.840177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831228+0000","last_change":"2024-04-26T21:30:53.837657+0000","last_active":"2024-04-26T21:31:21.831228+0000","last_peered":"2024-04-26T21:31:21.831228+0000","last_clean":"2024-04-26T21:31:21.831228+0000","last_became_active":"2024-04-26T21:30:53.837354+0000","last_became_peered":"2024-04-26T21:30:53.837354+0000","last_unstale":"2024-04-26T21:31:21.831228+0000","last_undegraded":"2024-04-26T21:31:21.831228+0000","last_fullsized":"2024-04-26T21:31:21.831228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:34.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827994+0000","last_change":"2024-04-26T21:30:53.811372+0000","last_active":"2024-04-26T21:31:21.827994+0000","last_peered":"2024-04-26T21:31:21.827994+0000","last_clean":"2024-04-26T21:31:21.827994+0000","last_became_active":"2024-04-26T21:30:53.810124+0000","last_became_peered":"2024-04-26T21:30:53.810124+0000","last_unstale":"2024-04-26T21:31:21.827994+0000","last_undegraded":"2024-04-26T21:31:21.827994+0000","last_fullsized":"2024-04-26T21:31:21.827994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:14.134041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.037154+0000","last_change":"2024-04-26T21:31:22.037154+0000","last_active":"2024-04-26T21:31:22.037154+0000","last_peered":"2024-04-26T21:31:22.037154+0000","last_clean":"2024-04-26T21:31:22.037154+0000","last_became_active":"2024-04-26T21:31:22.035817+0000","last_became_peered":"2024-04-26T21:31:22.035817+0000","last_unstale":"2024-04-26T21:31:22.037154+0000","last_undegraded":"2024-04-26T21:31:22.037154+0000","last_fullsized":"2024-04-26T21:31:22.037154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:53.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59839,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":809,"num_read_kb":469,"num_write":236,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":179517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":470384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547240,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1113545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":469811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972407,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":470077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1063098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":469528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":522685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":471258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":523575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":469353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":469892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691916,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1062385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:32:23.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:32:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:23 smithi097 ceph-mon[28127]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:23.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:23 smithi097 ceph-mon[28127]: pgmap v411: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:23 smithi143 ceph-mon[31461]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:23 smithi143 ceph-mon[31461]: pgmap v411: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:23 smithi111 ceph-mon[31001]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:23 smithi111 ceph-mon[31001]: pgmap v411: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:25.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:25 smithi097 ceph-mon[28127]: pgmap v412: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:25 smithi143 ceph-mon[31461]: pgmap v412: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.069+0000 7f897d000700 1 -- 172.21.15.97:0/2941726683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806bad0 msgr2=0x7f89781102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.069+0000 7f897d000700 1 --2- 172.21.15.97:0/2941726683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806bad0 0x7f89781102e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f897000b0a0 tx=0x7f897000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 -- 172.21.15.97:0/2941726683 shutdown_connections 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 --2- 172.21.15.97:0/2941726683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806bad0 0x7f89781102e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 --2- 172.21.15.97:0/2941726683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 0x7f8978109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 --2- 172.21.15.97:0/2941726683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806b130 0x7f897806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 -- 172.21.15.97:0/2941726683 >> 172.21.15.97:0/2941726683 conn(0x7f89780792d0 msgr2=0x7f89780796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 -- 172.21.15.97:0/2941726683 shutdown_connections 2024-04-26T21:32:26.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.070+0000 7f897d000700 1 -- 172.21.15.97:0/2941726683 wait complete. 2024-04-26T21:32:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.071+0000 7f897d000700 1 Processor -- start 2024-04-26T21:32:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.071+0000 7f897d000700 1 -- start start 2024-04-26T21:32:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 0x7f897819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 0x7f8978199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8978113470 con 0x7f897806b130 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89781132f0 con 0x7f897806bad0 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f897d000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f89781135f0 con 0x7f89781094a0 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f8975e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f8975e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59698/0 (socket says 172.21.15.97:59698) 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f8975e00700 1 -- 172.21.15.97:0/3402278623 learned_addr learned my addr 172.21.15.97:0/3402278623 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f896d400700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 0x7f897819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.072+0000 7f8976800700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 0x7f8978199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 -- 172.21.15.97:0/3402278623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 msgr2=0x7f8978199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:26.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 0x7f8978199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 -- 172.21.15.97:0/3402278623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 msgr2=0x7f897819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 0x7f897819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 -- 172.21.15.97:0/3402278623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897819a050 con 0x7f897806b130 2024-04-26T21:32:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f896d400700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 0x7f897819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f8975e00700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f896800d980 tx=0x7f896800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968017070 con 0x7f897806b130 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897819a340 con 0x7f897806b130 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8968004830 con 0x7f897806b130 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680053a0 con 0x7f897806b130 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.073+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a6620 con 0x7f897806b130 2024-04-26T21:32:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.075+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8978104b50 con 0x7f897806b130 2024-04-26T21:32:26.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.076+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89680049e0 con 0x7f897806b130 2024-04-26T21:32:26.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.078+0000 7f896f400700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 0x7f895c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:26.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.078+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f89680d0430 con 0x7f897806b130 2024-04-26T21:32:26.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.078+0000 7f896d400700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 0x7f895c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:26.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.079+0000 7f896d400700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 0x7f895c0a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8960005fd0 tx=0x7f8960005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:26.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.083+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8968020020 con 0x7f897806b130 2024-04-26T21:32:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:25 smithi111 ceph-mon[31001]: pgmap v412: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:26.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.308+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f897819aca0 con 0x7f895c0a3d50 2024-04-26T21:32:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.338+0000 7f896f400700 1 -- 172.21.15.97:0/3402278623 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258799 (secure 0 0 0) 0x7f897819aca0 con 0x7f895c0a3d50 2024-04-26T21:32:26.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 msgr2=0x7f895c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 0x7f895c0a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8960005fd0 tx=0x7f8960005e30 comp rx=0 tx=0).stop 2024-04-26T21:32:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 msgr2=0x7f897819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f896800d980 tx=0x7f896800dd40 comp rx=0 tx=0).stop 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 shutdown_connections 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f895c0a3d50 0x7f895c0a6210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89781094a0 0x7f8978199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897806b130 0x7f897819f6a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 --2- 172.21.15.97:0/3402278623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f897806bad0 0x7f897819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 >> 172.21.15.97:0/3402278623 conn(0x7f89780792d0 msgr2=0x7f89781002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.345+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 shutdown_connections 2024-04-26T21:32:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:26.346+0000 7f897d000700 1 -- 172.21.15.97:0/3402278623 wait complete. 2024-04-26T21:32:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-26T21:32:27.195 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-04-26T21:32:24.686352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650441,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":970,"num_read_kb":598,"num_write":419,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392004,"kb_used_data":13668,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549820,"statfs":{"total":1151940427776,"available":1151539015680,"internally_reserved":0,"allocated":13996032,"data_stored":7575591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004591"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.838455+0000","last_change":"2024-04-26T21:30:53.833872+0000","last_active":"2024-04-26T21:31:21.838455+0000","last_peered":"2024-04-26T21:31:21.838455+0000","last_clean":"2024-04-26T21:31:21.838455+0000","last_became_active":"2024-04-26T21:30:53.832109+0000","last_became_peered":"2024-04-26T21:30:53.832109+0000","last_unstale":"2024-04-26T21:31:21.838455+0000","last_undegraded":"2024-04-26T21:31:21.838455+0000","last_fullsized":"2024-04-26T21:31:21.838455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:50.076505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844785+0000","last_change":"2024-04-26T21:30:53.827034+0000","last_active":"2024-04-26T21:31:21.844785+0000","last_peered":"2024-04-26T21:31:21.844785+0000","last_clean":"2024-04-26T21:31:21.844785+0000","last_became_active":"2024-04-26T21:30:53.826682+0000","last_became_peered":"2024-04-26T21:30:53.826682+0000","last_unstale":"2024-04-26T21:31:21.844785+0000","last_undegraded":"2024-04-26T21:31:21.844785+0000","last_fullsized":"2024-04-26T21:31:21.844785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:39.445720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844320+0000","last_change":"2024-04-26T21:30:53.833196+0000","last_active":"2024-04-26T21:31:21.844320+0000","last_peered":"2024-04-26T21:31:21.844320+0000","last_clean":"2024-04-26T21:31:21.844320+0000","last_became_active":"2024-04-26T21:30:53.831214+0000","last_became_peered":"2024-04-26T21:30:53.831214+0000","last_unstale":"2024-04-26T21:31:21.844320+0000","last_undegraded":"2024-04-26T21:31:21.844320+0000","last_fullsized":"2024-04-26T21:31:21.844320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:17.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.082060+0000","last_change":"2024-04-26T21:30:53.829249+0000","last_active":"2024-04-26T21:31:22.082060+0000","last_peered":"2024-04-26T21:31:22.082060+0000","last_clean":"2024-04-26T21:31:22.082060+0000","last_became_active":"2024-04-26T21:30:53.828170+0000","last_became_peered":"2024-04-26T21:30:53.828170+0000","last_unstale":"2024-04-26T21:31:22.082060+0000","last_undegraded":"2024-04-26T21:31:22.082060+0000","last_fullsized":"2024-04-26T21:31:22.082060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:30.729567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.847479+0000","last_change":"2024-04-26T21:30:53.824137+0000","last_active":"2024-04-26T21:31:21.847479+0000","last_peered":"2024-04-26T21:31:21.847479+0000","last_clean":"2024-04-26T21:31:21.847479+0000","last_became_active":"2024-04-26T21:30:53.822807+0000","last_became_peered":"2024-04-26T21:30:53.822807+0000","last_unstale":"2024-04-26T21:31:21.847479+0000","last_undegraded":"2024-04-26T21:31:21.847479+0000","last_fullsized":"2024-04-26T21:31:21.847479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:03.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.947619+0000","last_change":"2024-04-26T21:30:53.835932+0000","last_active":"2024-04-26T21:31:21.947619+0000","last_peered":"2024-04-26T21:31:21.947619+0000","last_clean":"2024-04-26T21:31:21.947619+0000","last_became_active":"2024-04-26T21:30:53.835500+0000","last_became_peered":"2024-04-26T21:30:53.835500+0000","last_unstale":"2024-04-26T21:31:21.947619+0000","last_undegraded":"2024-04-26T21:31:21.947619+0000","last_fullsized":"2024-04-26T21:31:21.947619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:34.312999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519509+0000","last_change":"2024-04-26T21:30:53.815175+0000","last_active":"2024-04-26T21:31:22.519509+0000","last_peered":"2024-04-26T21:31:22.519509+0000","last_clean":"2024-04-26T21:31:22.519509+0000","last_became_active":"2024-04-26T21:30:53.814774+0000","last_became_peered":"2024-04-26T21:30:53.814774+0000","last_unstale":"2024-04-26T21:31:22.519509+0000","last_undegraded":"2024-04-26T21:31:22.519509+0000","last_fullsized":"2024-04-26T21:31:22.519509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:21.815872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.841837+0000","last_change":"2024-04-26T21:30:53.824718+0000","last_active":"2024-04-26T21:31:21.841837+0000","last_peered":"2024-04-26T21:31:21.841837+0000","last_clean":"2024-04-26T21:31:21.841837+0000","last_became_active":"2024-04-26T21:30:53.824372+0000","last_became_peered":"2024-04-26T21:30:53.824372+0000","last_unstale":"2024-04-26T21:31:21.841837+0000","last_undegraded":"2024-04-26T21:31:21.841837+0000","last_fullsized":"2024-04-26T21:31:21.841837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:33.450906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831963+0000","last_change":"2024-04-26T21:30:53.830237+0000","last_active":"2024-04-26T21:31:21.831963+0000","last_peered":"2024-04-26T21:31:21.831963+0000","last_clean":"2024-04-26T21:31:21.831963+0000","last_became_active":"2024-04-26T21:30:53.829837+0000","last_became_peered":"2024-04-26T21:30:53.829837+0000","last_unstale":"2024-04-26T21:31:21.831963+0000","last_undegraded":"2024-04-26T21:31:21.831963+0000","last_fullsized":"2024-04-26T21:31:21.831963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:33:15.161046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948554+0000","last_change":"2024-04-26T21:30:53.838240+0000","last_active":"2024-04-26T21:31:21.948554+0000","last_peered":"2024-04-26T21:31:21.948554+0000","last_clean":"2024-04-26T21:31:21.948554+0000","last_became_active":"2024-04-26T21:30:53.837741+0000","last_became_peered":"2024-04-26T21:30:53.837741+0000","last_unstale":"2024-04-26T21:31:21.948554+0000","last_undegraded":"2024-04-26T21:31:21.948554+0000","last_fullsized":"2024-04-26T21:31:21.948554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:33.879043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520424+0000","last_change":"2024-04-26T21:30:53.830564+0000","last_active":"2024-04-26T21:31:22.520424+0000","last_peered":"2024-04-26T21:31:22.520424+0000","last_clean":"2024-04-26T21:31:22.520424+0000","last_became_active":"2024-04-26T21:30:53.829967+0000","last_became_peered":"2024-04-26T21:30:53.829967+0000","last_unstale":"2024-04-26T21:31:22.520424+0000","last_undegraded":"2024-04-26T21:31:22.520424+0000","last_fullsized":"2024-04-26T21:31:22.520424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:06.931219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.081769+0000","last_change":"2024-04-26T21:30:53.829434+0000","last_active":"2024-04-26T21:31:22.081769+0000","last_peered":"2024-04-26T21:31:22.081769+0000","last_clean":"2024-04-26T21:31:22.081769+0000","last_became_active":"2024-04-26T21:30:53.828657+0000","last_became_peered":"2024-04-26T21:30:53.828657+0000","last_unstale":"2024-04-26T21:31:22.081769+0000","last_undegraded":"2024-04-26T21:31:22.081769+0000","last_fullsized":"2024-04-26T21:31:22.081769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:12.215856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.035994+0000","last_change":"2024-04-26T21:31:22.035994+0000","last_active":"2024-04-26T21:31:22.035994+0000","last_peered":"2024-04-26T21:31:22.035994+0000","last_clean":"2024-04-26T21:31:22.035994+0000","last_became_active":"2024-04-26T21:31:22.034795+0000","last_became_peered":"2024-04-26T21:31:22.034795+0000","last_unstale":"2024-04-26T21:31:22.035994+0000","last_undegraded":"2024-04-26T21:31:22.035994+0000","last_fullsized":"2024-04-26T21:31:22.035994+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:57.990920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843951+0000","last_change":"2024-04-26T21:30:53.824573+0000","last_active":"2024-04-26T21:31:21.843951+0000","last_peered":"2024-04-26T21:31:21.843951+0000","last_clean":"2024-04-26T21:31:21.843951+0000","last_became_active":"2024-04-26T21:30:53.824242+0000","last_became_peered":"2024-04-26T21:30:53.824242+0000","last_unstale":"2024-04-26T21:31:21.843951+0000","last_undegraded":"2024-04-26T21:31:21.843951+0000","last_fullsized":"2024-04-26T21:31:21.843951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:12.271919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843631+0000","last_change":"2024-04-26T21:30:53.827619+0000","last_active":"2024-04-26T21:31:21.843631+0000","last_peered":"2024-04-26T21:31:21.843631+0000","last_clean":"2024-04-26T21:31:21.843631+0000","last_became_active":"2024-04-26T21:30:53.827301+0000","last_became_peered":"2024-04-26T21:30:53.827301+0000","last_unstale":"2024-04-26T21:31:21.843631+0000","last_undegraded":"2024-04-26T21:31:21.843631+0000","last_fullsized":"2024-04-26T21:31:21.843631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:25.152302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.830809+0000","last_change":"2024-04-26T21:30:53.825785+0000","last_active":"2024-04-26T21:31:21.830809+0000","last_peered":"2024-04-26T21:31:21.830809+0000","last_clean":"2024-04-26T21:31:21.830809+0000","last_became_active":"2024-04-26T21:30:53.821673+0000","last_became_peered":"2024-04-26T21:30:53.821673+0000","last_unstale":"2024-04-26T21:31:21.830809+0000","last_undegraded":"2024-04-26T21:31:21.830809+0000","last_fullsized":"2024-04-26T21:31:21.830809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:39.138995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.839087+0000","last_change":"2024-04-26T21:30:53.834768+0000","last_active":"2024-04-26T21:31:21.839087+0000","last_peered":"2024-04-26T21:31:21.839087+0000","last_clean":"2024-04-26T21:31:21.839087+0000","last_became_active":"2024-04-26T21:30:53.831624+0000","last_became_peered":"2024-04-26T21:30:53.831624+0000","last_unstale":"2024-04-26T21:31:21.839087+0000","last_undegraded":"2024-04-26T21:31:21.839087+0000","last_fullsized":"2024-04-26T21:31:21.839087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:49.228987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033911+0000","last_change":"2024-04-26T21:30:53.828348+0000","last_active":"2024-04-26T21:31:22.033911+0000","last_peered":"2024-04-26T21:31:22.033911+0000","last_clean":"2024-04-26T21:31:22.033911+0000","last_became_active":"2024-04-26T21:30:53.827926+0000","last_became_peered":"2024-04-26T21:30:53.827926+0000","last_unstale":"2024-04-26T21:31:22.033911+0000","last_undegraded":"2024-04-26T21:31:22.033911+0000","last_fullsized":"2024-04-26T21:31:22.033911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.845547+0000","last_change":"2024-04-26T21:30:53.824205+0000","last_active":"2024-04-26T21:31:21.845547+0000","last_peered":"2024-04-26T21:31:21.845547+0000","last_clean":"2024-04-26T21:31:21.845547+0000","last_became_active":"2024-04-26T21:30:53.823787+0000","last_became_peered":"2024-04-26T21:30:53.823787+0000","last_unstale":"2024-04-26T21:31:21.845547+0000","last_undegraded":"2024-04-26T21:31:21.845547+0000","last_fullsized":"2024-04-26T21:31:21.845547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:23.325924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:18.918525+0000","last_change":"2024-04-26T21:30:52.803835+0000","last_active":"2024-04-26T21:32:18.918525+0000","last_peered":"2024-04-26T21:32:18.918525+0000","last_clean":"2024-04-26T21:32:18.918525+0000","last_became_active":"2024-04-26T21:30:52.803541+0000","last_became_peered":"2024-04-26T21:30:52.803541+0000","last_unstale":"2024-04-26T21:32:18.918525+0000","last_undegraded":"2024-04-26T21:32:18.918525+0000","last_fullsized":"2024-04-26T21:32:18.918525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:16.314776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842833+0000","last_change":"2024-04-26T21:30:53.823811+0000","last_active":"2024-04-26T21:31:21.842833+0000","last_peered":"2024-04-26T21:31:21.842833+0000","last_clean":"2024-04-26T21:31:21.842833+0000","last_became_active":"2024-04-26T21:30:53.823284+0000","last_became_peered":"2024-04-26T21:30:53.823284+0000","last_unstale":"2024-04-26T21:31:21.842833+0000","last_undegraded":"2024-04-26T21:31:21.842833+0000","last_fullsized":"2024-04-26T21:31:21.842833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:05.627445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.843011+0000","last_change":"2024-04-26T21:30:52.810994+0000","last_active":"2024-04-26T21:31:21.843011+0000","last_peered":"2024-04-26T21:31:21.843011+0000","last_clean":"2024-04-26T21:31:21.843011+0000","last_became_active":"2024-04-26T21:30:52.809509+0000","last_became_peered":"2024-04-26T21:30:52.809509+0000","last_unstale":"2024-04-26T21:31:21.843011+0000","last_undegraded":"2024-04-26T21:31:21.843011+0000","last_fullsized":"2024-04-26T21:31:21.843011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:24.366289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844835+0000","last_change":"2024-04-26T21:30:53.830471+0000","last_active":"2024-04-26T21:31:21.844835+0000","last_peered":"2024-04-26T21:31:21.844835+0000","last_clean":"2024-04-26T21:31:21.844835+0000","last_became_active":"2024-04-26T21:30:53.829841+0000","last_became_peered":"2024-04-26T21:30:53.829841+0000","last_unstale":"2024-04-26T21:31:21.844835+0000","last_undegraded":"2024-04-26T21:31:21.844835+0000","last_fullsized":"2024-04-26T21:31:21.844835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:25.988137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.849302+0000","last_change":"2024-04-26T21:30:52.812564+0000","last_active":"2024-04-26T21:31:21.849302+0000","last_peered":"2024-04-26T21:31:21.849302+0000","last_clean":"2024-04-26T21:31:21.849302+0000","last_became_active":"2024-04-26T21:30:52.811090+0000","last_became_peered":"2024-04-26T21:30:52.811090+0000","last_unstale":"2024-04-26T21:31:21.849302+0000","last_undegraded":"2024-04-26T21:31:21.849302+0000","last_fullsized":"2024-04-26T21:31:21.849302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:51.522689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831380+0000","last_change":"2024-04-26T21:30:53.828398+0000","last_active":"2024-04-26T21:31:21.831380+0000","last_peered":"2024-04-26T21:31:21.831380+0000","last_clean":"2024-04-26T21:31:21.831380+0000","last_became_active":"2024-04-26T21:30:53.827920+0000","last_became_peered":"2024-04-26T21:30:53.827920+0000","last_unstale":"2024-04-26T21:31:21.831380+0000","last_undegraded":"2024-04-26T21:31:21.831380+0000","last_fullsized":"2024-04-26T21:31:21.831380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:06.463121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848430+0000","last_change":"2024-04-26T21:30:52.798297+0000","last_active":"2024-04-26T21:31:21.848430+0000","last_peered":"2024-04-26T21:31:21.848430+0000","last_clean":"2024-04-26T21:31:21.848430+0000","last_became_active":"2024-04-26T21:30:52.797998+0000","last_became_peered":"2024-04-26T21:30:52.797998+0000","last_unstale":"2024-04-26T21:31:21.848430+0000","last_undegraded":"2024-04-26T21:31:21.848430+0000","last_fullsized":"2024-04-26T21:31:21.848430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:40.790353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832850+0000","last_change":"2024-04-26T21:30:53.830846+0000","last_active":"2024-04-26T21:31:21.832850+0000","last_peered":"2024-04-26T21:31:21.832850+0000","last_clean":"2024-04-26T21:31:21.832850+0000","last_became_active":"2024-04-26T21:30:53.830294+0000","last_became_peered":"2024-04-26T21:30:53.830294+0000","last_unstale":"2024-04-26T21:31:21.832850+0000","last_undegraded":"2024-04-26T21:31:21.832850+0000","last_fullsized":"2024-04-26T21:31:21.832850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:50.929819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":109,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:20.947928+0000","last_change":"2024-04-26T21:30:52.807155+0000","last_active":"2024-04-26T21:32:20.947928+0000","last_peered":"2024-04-26T21:32:20.947928+0000","last_clean":"2024-04-26T21:32:20.947928+0000","last_became_active":"2024-04-26T21:30:52.805230+0000","last_became_peered":"2024-04-26T21:30:52.805230+0000","last_unstale":"2024-04-26T21:32:20.947928+0000","last_undegraded":"2024-04-26T21:32:20.947928+0000","last_fullsized":"2024-04-26T21:32:20.947928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:44.478300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848460+0000","last_change":"2024-04-26T21:30:53.810334+0000","last_active":"2024-04-26T21:31:21.848460+0000","last_peered":"2024-04-26T21:31:21.848460+0000","last_clean":"2024-04-26T21:31:21.848460+0000","last_became_active":"2024-04-26T21:30:53.809867+0000","last_became_peered":"2024-04-26T21:30:53.809867+0000","last_unstale":"2024-04-26T21:31:21.848460+0000","last_undegraded":"2024-04-26T21:31:21.848460+0000","last_fullsized":"2024-04-26T21:31:21.848460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:46.377262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.947127+0000","last_change":"2024-04-26T21:30:52.804775+0000","last_active":"2024-04-26T21:31:21.947127+0000","last_peered":"2024-04-26T21:31:21.947127+0000","last_clean":"2024-04-26T21:31:21.947127+0000","last_became_active":"2024-04-26T21:30:52.804319+0000","last_became_peered":"2024-04-26T21:30:52.804319+0000","last_unstale":"2024-04-26T21:31:21.947127+0000","last_undegraded":"2024-04-26T21:31:21.947127+0000","last_fullsized":"2024-04-26T21:31:21.947127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:12.761080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832486+0000","last_change":"2024-04-26T21:30:53.823021+0000","last_active":"2024-04-26T21:31:21.832486+0000","last_peered":"2024-04-26T21:31:21.832486+0000","last_clean":"2024-04-26T21:31:21.832486+0000","last_became_active":"2024-04-26T21:30:53.822633+0000","last_became_peered":"2024-04-26T21:30:53.822633+0000","last_unstale":"2024-04-26T21:31:21.832486+0000","last_undegraded":"2024-04-26T21:31:21.832486+0000","last_fullsized":"2024-04-26T21:31:21.832486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:32.178472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827806+0000","last_change":"2024-04-26T21:30:52.796729+0000","last_active":"2024-04-26T21:31:21.827806+0000","last_peered":"2024-04-26T21:31:21.827806+0000","last_clean":"2024-04-26T21:31:21.827806+0000","last_became_active":"2024-04-26T21:30:52.796341+0000","last_became_peered":"2024-04-26T21:30:52.796341+0000","last_unstale":"2024-04-26T21:31:21.827806+0000","last_undegraded":"2024-04-26T21:31:21.827806+0000","last_fullsized":"2024-04-26T21:31:21.827806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:30.434541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.841762+0000","last_change":"2024-04-26T21:30:53.816136+0000","last_active":"2024-04-26T21:31:21.841762+0000","last_peered":"2024-04-26T21:31:21.841762+0000","last_clean":"2024-04-26T21:31:21.841762+0000","last_became_active":"2024-04-26T21:30:53.815758+0000","last_became_peered":"2024-04-26T21:30:53.815758+0000","last_unstale":"2024-04-26T21:31:21.841762+0000","last_undegraded":"2024-04-26T21:31:21.841762+0000","last_fullsized":"2024-04-26T21:31:21.841762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:41.606483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831981+0000","last_change":"2024-04-26T21:30:52.804102+0000","last_active":"2024-04-26T21:31:21.831981+0000","last_peered":"2024-04-26T21:31:21.831981+0000","last_clean":"2024-04-26T21:31:21.831981+0000","last_became_active":"2024-04-26T21:30:52.803808+0000","last_became_peered":"2024-04-26T21:30:52.803808+0000","last_unstale":"2024-04-26T21:31:21.831981+0000","last_undegraded":"2024-04-26T21:31:21.831981+0000","last_fullsized":"2024-04-26T21:31:21.831981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:16.357651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033354+0000","last_change":"2024-04-26T21:30:53.822475+0000","last_active":"2024-04-26T21:31:22.033354+0000","last_peered":"2024-04-26T21:31:22.033354+0000","last_clean":"2024-04-26T21:31:22.033354+0000","last_became_active":"2024-04-26T21:30:53.822077+0000","last_became_peered":"2024-04-26T21:30:53.822077+0000","last_unstale":"2024-04-26T21:31:22.033354+0000","last_undegraded":"2024-04-26T21:31:22.033354+0000","last_fullsized":"2024-04-26T21:31:22.033354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:09.289557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":185,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:20.946150+0000","last_change":"2024-04-26T21:30:52.802080+0000","last_active":"2024-04-26T21:32:20.946150+0000","last_peered":"2024-04-26T21:32:20.946150+0000","last_clean":"2024-04-26T21:32:20.946150+0000","last_became_active":"2024-04-26T21:30:52.801645+0000","last_became_peered":"2024-04-26T21:30:52.801645+0000","last_unstale":"2024-04-26T21:32:20.946150+0000","last_undegraded":"2024-04-26T21:32:20.946150+0000","last_fullsized":"2024-04-26T21:32:20.946150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:00.213924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":162,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832072+0000","last_change":"2024-04-26T21:30:53.832936+0000","last_active":"2024-04-26T21:31:21.832072+0000","last_peered":"2024-04-26T21:31:21.832072+0000","last_clean":"2024-04-26T21:31:21.832072+0000","last_became_active":"2024-04-26T21:30:53.830821+0000","last_became_peered":"2024-04-26T21:30:53.830821+0000","last_unstale":"2024-04-26T21:31:21.832072+0000","last_undegraded":"2024-04-26T21:31:21.832072+0000","last_fullsized":"2024-04-26T21:31:21.832072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:08.540653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520227+0000","last_change":"2024-04-26T21:30:52.812687+0000","last_active":"2024-04-26T21:31:22.520227+0000","last_peered":"2024-04-26T21:31:22.520227+0000","last_clean":"2024-04-26T21:31:22.520227+0000","last_became_active":"2024-04-26T21:30:52.812055+0000","last_became_peered":"2024-04-26T21:30:52.812055+0000","last_unstale":"2024-04-26T21:31:22.520227+0000","last_undegraded":"2024-04-26T21:31:22.520227+0000","last_fullsized":"2024-04-26T21:31:22.520227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:51.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844669+0000","last_change":"2024-04-26T21:30:53.829887+0000","last_active":"2024-04-26T21:31:21.844669+0000","last_peered":"2024-04-26T21:31:21.844669+0000","last_clean":"2024-04-26T21:31:21.844669+0000","last_became_active":"2024-04-26T21:30:53.829355+0000","last_became_peered":"2024-04-26T21:30:53.829355+0000","last_unstale":"2024-04-26T21:31:21.844669+0000","last_undegraded":"2024-04-26T21:31:21.844669+0000","last_fullsized":"2024-04-26T21:31:21.844669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:49.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.833113+0000","last_change":"2024-04-26T21:30:52.808859+0000","last_active":"2024-04-26T21:31:21.833113+0000","last_peered":"2024-04-26T21:31:21.833113+0000","last_clean":"2024-04-26T21:31:21.833113+0000","last_became_active":"2024-04-26T21:30:52.807232+0000","last_became_peered":"2024-04-26T21:30:52.807232+0000","last_unstale":"2024-04-26T21:31:21.833113+0000","last_undegraded":"2024-04-26T21:31:21.833113+0000","last_fullsized":"2024-04-26T21:31:21.833113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:44.492893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831743+0000","last_change":"2024-04-26T21:30:53.829723+0000","last_active":"2024-04-26T21:31:21.831743+0000","last_peered":"2024-04-26T21:31:21.831743+0000","last_clean":"2024-04-26T21:31:21.831743+0000","last_became_active":"2024-04-26T21:30:53.827799+0000","last_became_peered":"2024-04-26T21:30:53.827799+0000","last_unstale":"2024-04-26T21:31:21.831743+0000","last_undegraded":"2024-04-26T21:31:21.831743+0000","last_fullsized":"2024-04-26T21:31:21.831743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:10:20.798100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":215,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:20.954242+0000","last_change":"2024-04-26T21:30:52.812217+0000","last_active":"2024-04-26T21:32:20.954242+0000","last_peered":"2024-04-26T21:32:20.954242+0000","last_clean":"2024-04-26T21:32:20.954242+0000","last_became_active":"2024-04-26T21:30:52.811730+0000","last_became_peered":"2024-04-26T21:30:52.811730+0000","last_unstale":"2024-04-26T21:32:20.954242+0000","last_undegraded":"2024-04-26T21:32:20.954242+0000","last_fullsized":"2024-04-26T21:32:20.954242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:08.012161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52244,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":135,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848793+0000","last_change":"2024-04-26T21:30:53.813388+0000","last_active":"2024-04-26T21:31:21.848793+0000","last_peered":"2024-04-26T21:31:21.848793+0000","last_clean":"2024-04-26T21:31:21.848793+0000","last_became_active":"2024-04-26T21:30:53.812972+0000","last_became_peered":"2024-04-26T21:30:53.812972+0000","last_unstale":"2024-04-26T21:31:21.848793+0000","last_undegraded":"2024-04-26T21:31:21.848793+0000","last_fullsized":"2024-04-26T21:31:21.848793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:22.480641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844911+0000","last_change":"2024-04-26T21:30:52.804426+0000","last_active":"2024-04-26T21:31:21.844911+0000","last_peered":"2024-04-26T21:31:21.844911+0000","last_clean":"2024-04-26T21:31:21.844911+0000","last_became_active":"2024-04-26T21:30:52.804083+0000","last_became_peered":"2024-04-26T21:30:52.804083+0000","last_unstale":"2024-04-26T21:31:21.844911+0000","last_undegraded":"2024-04-26T21:31:21.844911+0000","last_fullsized":"2024-04-26T21:31:21.844911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:10.164976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827875+0000","last_change":"2024-04-26T21:30:53.829326+0000","last_active":"2024-04-26T21:31:21.827875+0000","last_peered":"2024-04-26T21:31:21.827875+0000","last_clean":"2024-04-26T21:31:21.827875+0000","last_became_active":"2024-04-26T21:30:53.828863+0000","last_became_peered":"2024-04-26T21:30:53.828863+0000","last_unstale":"2024-04-26T21:31:21.827875+0000","last_undegraded":"2024-04-26T21:31:21.827875+0000","last_fullsized":"2024-04-26T21:31:21.827875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:56.756187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831957+0000","last_change":"2024-04-26T21:30:52.805302+0000","last_active":"2024-04-26T21:31:21.831957+0000","last_peered":"2024-04-26T21:31:21.831957+0000","last_clean":"2024-04-26T21:31:21.831957+0000","last_became_active":"2024-04-26T21:30:52.804869+0000","last_became_peered":"2024-04-26T21:30:52.804869+0000","last_unstale":"2024-04-26T21:31:21.831957+0000","last_undegraded":"2024-04-26T21:31:21.831957+0000","last_fullsized":"2024-04-26T21:31:21.831957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:21.299474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519967+0000","last_change":"2024-04-26T21:30:53.823827+0000","last_active":"2024-04-26T21:31:22.519967+0000","last_peered":"2024-04-26T21:31:22.519967+0000","last_clean":"2024-04-26T21:31:22.519967+0000","last_became_active":"2024-04-26T21:30:53.823492+0000","last_became_peered":"2024-04-26T21:30:53.823492+0000","last_unstale":"2024-04-26T21:31:22.519967+0000","last_undegraded":"2024-04-26T21:31:22.519967+0000","last_fullsized":"2024-04-26T21:31:22.519967+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.514753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":106,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:17.942971+0000","last_change":"2024-04-26T21:30:52.801468+0000","last_active":"2024-04-26T21:32:17.942971+0000","last_peered":"2024-04-26T21:32:17.942971+0000","last_clean":"2024-04-26T21:32:17.942971+0000","last_became_active":"2024-04-26T21:30:52.801071+0000","last_became_peered":"2024-04-26T21:30:52.801071+0000","last_unstale":"2024-04-26T21:32:17.942971+0000","last_undegraded":"2024-04-26T21:32:17.942971+0000","last_fullsized":"2024-04-26T21:32:17.942971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:52.492913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519357+0000","last_change":"2024-04-26T21:30:53.824075+0000","last_active":"2024-04-26T21:31:22.519357+0000","last_peered":"2024-04-26T21:31:22.519357+0000","last_clean":"2024-04-26T21:31:22.519357+0000","last_became_active":"2024-04-26T21:30:53.823753+0000","last_became_peered":"2024-04-26T21:30:53.823753+0000","last_unstale":"2024-04-26T21:31:22.519357+0000","last_undegraded":"2024-04-26T21:31:22.519357+0000","last_fullsized":"2024-04-26T21:31:22.519357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:44:09.294481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.154980+0000","last_change":"2024-04-26T21:31:22.153880+0000","last_active":"2024-04-26T21:31:22.154980+0000","last_peered":"2024-04-26T21:31:22.154980+0000","last_clean":"2024-04-26T21:31:22.154980+0000","last_became_active":"2024-04-26T21:31:22.085354+0000","last_became_peered":"2024-04-26T21:31:22.085354+0000","last_unstale":"2024-04-26T21:31:22.154980+0000","last_undegraded":"2024-04-26T21:31:22.154980+0000","last_fullsized":"2024-04-26T21:31:22.154980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:05.687450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831958+0000","last_change":"2024-04-26T21:30:53.835273+0000","last_active":"2024-04-26T21:31:21.831958+0000","last_peered":"2024-04-26T21:31:21.831958+0000","last_clean":"2024-04-26T21:31:21.831958+0000","last_became_active":"2024-04-26T21:30:53.834922+0000","last_became_peered":"2024-04-26T21:30:53.834922+0000","last_unstale":"2024-04-26T21:31:21.831958+0000","last_undegraded":"2024-04-26T21:31:21.831958+0000","last_fullsized":"2024-04-26T21:31:21.831958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:43.992579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848829+0000","last_change":"2024-04-26T21:30:52.797639+0000","last_active":"2024-04-26T21:31:21.848829+0000","last_peered":"2024-04-26T21:31:21.848829+0000","last_clean":"2024-04-26T21:31:21.848829+0000","last_became_active":"2024-04-26T21:30:52.796576+0000","last_became_peered":"2024-04-26T21:30:52.796576+0000","last_unstale":"2024-04-26T21:31:21.848829+0000","last_undegraded":"2024-04-26T21:31:21.848829+0000","last_fullsized":"2024-04-26T21:31:21.848829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:33.730232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.838255+0000","last_change":"2024-04-26T21:30:53.836190+0000","last_active":"2024-04-26T21:31:21.838255+0000","last_peered":"2024-04-26T21:31:21.838255+0000","last_clean":"2024-04-26T21:31:21.838255+0000","last_became_active":"2024-04-26T21:30:53.835302+0000","last_became_peered":"2024-04-26T21:30:53.835302+0000","last_unstale":"2024-04-26T21:31:21.838255+0000","last_undegraded":"2024-04-26T21:31:21.838255+0000","last_fullsized":"2024-04-26T21:31:21.838255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:13.222578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033576+0000","last_change":"2024-04-26T21:30:52.809928+0000","last_active":"2024-04-26T21:31:22.033576+0000","last_peered":"2024-04-26T21:31:22.033576+0000","last_clean":"2024-04-26T21:31:22.033576+0000","last_became_active":"2024-04-26T21:30:52.808667+0000","last_became_peered":"2024-04-26T21:30:52.808667+0000","last_unstale":"2024-04-26T21:31:22.033576+0000","last_undegraded":"2024-04-26T21:31:22.033576+0000","last_fullsized":"2024-04-26T21:31:22.033576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:15.375105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948044+0000","last_change":"2024-04-26T21:30:53.836649+0000","last_active":"2024-04-26T21:31:21.948044+0000","last_peered":"2024-04-26T21:31:21.948044+0000","last_clean":"2024-04-26T21:31:21.948044+0000","last_became_active":"2024-04-26T21:30:53.836252+0000","last_became_peered":"2024-04-26T21:30:53.836252+0000","last_unstale":"2024-04-26T21:31:21.948044+0000","last_undegraded":"2024-04-26T21:31:21.948044+0000","last_fullsized":"2024-04-26T21:31:21.948044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:53.797872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":105,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:18.919570+0000","last_change":"2024-04-26T21:30:52.808196+0000","last_active":"2024-04-26T21:32:18.919570+0000","last_peered":"2024-04-26T21:32:18.919570+0000","last_clean":"2024-04-26T21:32:18.919570+0000","last_became_active":"2024-04-26T21:30:52.806445+0000","last_became_peered":"2024-04-26T21:30:52.806445+0000","last_unstale":"2024-04-26T21:32:18.919570+0000","last_undegraded":"2024-04-26T21:32:18.919570+0000","last_fullsized":"2024-04-26T21:32:18.919570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:39.815485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.948196+0000","last_change":"2024-04-26T21:30:53.837998+0000","last_active":"2024-04-26T21:31:21.948196+0000","last_peered":"2024-04-26T21:31:21.948196+0000","last_clean":"2024-04-26T21:31:21.948196+0000","last_became_active":"2024-04-26T21:30:53.837443+0000","last_became_peered":"2024-04-26T21:30:53.837443+0000","last_unstale":"2024-04-26T21:31:21.948196+0000","last_undegraded":"2024-04-26T21:31:21.948196+0000","last_fullsized":"2024-04-26T21:31:21.948196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:04.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.833579+0000","last_change":"2024-04-26T21:30:53.835501+0000","last_active":"2024-04-26T21:31:21.833579+0000","last_peered":"2024-04-26T21:31:21.833579+0000","last_clean":"2024-04-26T21:31:21.833579+0000","last_became_active":"2024-04-26T21:30:53.835096+0000","last_became_peered":"2024-04-26T21:30:53.835096+0000","last_unstale":"2024-04-26T21:31:21.833579+0000","last_undegraded":"2024-04-26T21:31:21.833579+0000","last_fullsized":"2024-04-26T21:31:21.833579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:26.178001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":108,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:21.952728+0000","last_change":"2024-04-26T21:30:52.805357+0000","last_active":"2024-04-26T21:32:21.952728+0000","last_peered":"2024-04-26T21:32:21.952728+0000","last_clean":"2024-04-26T21:32:21.952728+0000","last_became_active":"2024-04-26T21:30:52.804902+0000","last_became_peered":"2024-04-26T21:30:52.804902+0000","last_unstale":"2024-04-26T21:32:21.952728+0000","last_undegraded":"2024-04-26T21:32:21.952728+0000","last_fullsized":"2024-04-26T21:32:21.952728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:29.276141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842272+0000","last_change":"2024-04-26T21:30:53.835448+0000","last_active":"2024-04-26T21:31:21.842272+0000","last_peered":"2024-04-26T21:31:21.842272+0000","last_clean":"2024-04-26T21:31:21.842272+0000","last_became_active":"2024-04-26T21:30:53.835054+0000","last_became_peered":"2024-04-26T21:30:53.835054+0000","last_unstale":"2024-04-26T21:31:21.842272+0000","last_undegraded":"2024-04-26T21:31:21.842272+0000","last_fullsized":"2024-04-26T21:31:21.842272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:20:37.892713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.081775+0000","last_change":"2024-04-26T21:30:52.792942+0000","last_active":"2024-04-26T21:31:22.081775+0000","last_peered":"2024-04-26T21:31:22.081775+0000","last_clean":"2024-04-26T21:31:22.081775+0000","last_became_active":"2024-04-26T21:30:52.792553+0000","last_became_peered":"2024-04-26T21:30:52.792553+0000","last_unstale":"2024-04-26T21:31:22.081775+0000","last_undegraded":"2024-04-26T21:31:22.081775+0000","last_fullsized":"2024-04-26T21:31:22.081775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:10.915883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519688+0000","last_change":"2024-04-26T21:29:15.217076+0000","last_active":"2024-04-26T21:31:22.519688+0000","last_peered":"2024-04-26T21:31:22.519688+0000","last_clean":"2024-04-26T21:31:22.519688+0000","last_became_active":"2024-04-26T21:29:15.215786+0000","last_became_peered":"2024-04-26T21:29:15.215786+0000","last_unstale":"2024-04-26T21:31:22.519688+0000","last_undegraded":"2024-04-26T21:31:22.519688+0000","last_fullsized":"2024-04-26T21:31:22.519688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:24:37.649628+0000","last_clean_scrub_stamp":"2024-04-26T21:24:37.649628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:47.795571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827632+0000","last_change":"2024-04-26T21:30:53.835344+0000","last_active":"2024-04-26T21:31:21.827632+0000","last_peered":"2024-04-26T21:31:21.827632+0000","last_clean":"2024-04-26T21:31:21.827632+0000","last_became_active":"2024-04-26T21:30:53.834838+0000","last_became_peered":"2024-04-26T21:30:53.834838+0000","last_unstale":"2024-04-26T21:31:21.827632+0000","last_undegraded":"2024-04-26T21:31:21.827632+0000","last_fullsized":"2024-04-26T21:31:21.827632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:37.588955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519851+0000","last_change":"2024-04-26T21:30:52.808173+0000","last_active":"2024-04-26T21:31:22.519851+0000","last_peered":"2024-04-26T21:31:22.519851+0000","last_clean":"2024-04-26T21:31:22.519851+0000","last_became_active":"2024-04-26T21:30:52.807688+0000","last_became_peered":"2024-04-26T21:30:52.807688+0000","last_unstale":"2024-04-26T21:31:22.519851+0000","last_undegraded":"2024-04-26T21:31:22.519851+0000","last_fullsized":"2024-04-26T21:31:22.519851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:58.364896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.848703+0000","last_change":"2024-04-26T21:30:52.810654+0000","last_active":"2024-04-26T21:31:21.848703+0000","last_peered":"2024-04-26T21:31:21.848703+0000","last_clean":"2024-04-26T21:31:21.848703+0000","last_became_active":"2024-04-26T21:30:52.808959+0000","last_became_peered":"2024-04-26T21:30:52.808959+0000","last_unstale":"2024-04-26T21:31:21.848703+0000","last_undegraded":"2024-04-26T21:31:21.848703+0000","last_fullsized":"2024-04-26T21:31:21.848703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:59.134325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827988+0000","last_change":"2024-04-26T21:30:53.816961+0000","last_active":"2024-04-26T21:31:21.827988+0000","last_peered":"2024-04-26T21:31:21.827988+0000","last_clean":"2024-04-26T21:31:21.827988+0000","last_became_active":"2024-04-26T21:30:53.816571+0000","last_became_peered":"2024-04-26T21:30:53.816571+0000","last_unstale":"2024-04-26T21:31:21.827988+0000","last_undegraded":"2024-04-26T21:31:21.827988+0000","last_fullsized":"2024-04-26T21:31:21.827988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:43.218823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.033616+0000","last_change":"2024-04-26T21:30:52.801027+0000","last_active":"2024-04-26T21:31:22.033616+0000","last_peered":"2024-04-26T21:31:22.033616+0000","last_clean":"2024-04-26T21:31:22.033616+0000","last_became_active":"2024-04-26T21:30:52.797107+0000","last_became_peered":"2024-04-26T21:30:52.797107+0000","last_unstale":"2024-04-26T21:31:22.033616+0000","last_undegraded":"2024-04-26T21:31:22.033616+0000","last_fullsized":"2024-04-26T21:31:22.033616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:23.739673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842740+0000","last_change":"2024-04-26T21:30:53.821946+0000","last_active":"2024-04-26T21:31:21.842740+0000","last_peered":"2024-04-26T21:31:21.842740+0000","last_clean":"2024-04-26T21:31:21.842740+0000","last_became_active":"2024-04-26T21:30:53.821433+0000","last_became_peered":"2024-04-26T21:30:53.821433+0000","last_unstale":"2024-04-26T21:31:21.842740+0000","last_undegraded":"2024-04-26T21:31:21.842740+0000","last_fullsized":"2024-04-26T21:31:21.842740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:30.426334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.082756+0000","last_change":"2024-04-26T21:30:52.806302+0000","last_active":"2024-04-26T21:31:22.082756+0000","last_peered":"2024-04-26T21:31:22.082756+0000","last_clean":"2024-04-26T21:31:22.082756+0000","last_became_active":"2024-04-26T21:30:52.803402+0000","last_became_peered":"2024-04-26T21:30:52.803402+0000","last_unstale":"2024-04-26T21:31:22.082756+0000","last_undegraded":"2024-04-26T21:31:22.082756+0000","last_fullsized":"2024-04-26T21:31:22.082756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:07.163372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844868+0000","last_change":"2024-04-26T21:30:53.833803+0000","last_active":"2024-04-26T21:31:21.844868+0000","last_peered":"2024-04-26T21:31:21.844868+0000","last_clean":"2024-04-26T21:31:21.844868+0000","last_became_active":"2024-04-26T21:30:53.833378+0000","last_became_peered":"2024-04-26T21:30:53.833378+0000","last_unstale":"2024-04-26T21:31:21.844868+0000","last_undegraded":"2024-04-26T21:31:21.844868+0000","last_fullsized":"2024-04-26T21:31:21.844868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:14:58.505849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.830812+0000","last_change":"2024-04-26T21:30:52.810031+0000","last_active":"2024-04-26T21:31:21.830812+0000","last_peered":"2024-04-26T21:31:21.830812+0000","last_clean":"2024-04-26T21:31:21.830812+0000","last_became_active":"2024-04-26T21:30:52.809743+0000","last_became_peered":"2024-04-26T21:30:52.809743+0000","last_unstale":"2024-04-26T21:31:21.830812+0000","last_undegraded":"2024-04-26T21:31:21.830812+0000","last_fullsized":"2024-04-26T21:31:21.830812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:05.723934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.845313+0000","last_change":"2024-04-26T21:30:53.822563+0000","last_active":"2024-04-26T21:31:21.845313+0000","last_peered":"2024-04-26T21:31:21.845313+0000","last_clean":"2024-04-26T21:31:21.845313+0000","last_became_active":"2024-04-26T21:30:53.821848+0000","last_became_peered":"2024-04-26T21:30:53.821848+0000","last_unstale":"2024-04-26T21:31:21.845313+0000","last_undegraded":"2024-04-26T21:31:21.845313+0000","last_fullsized":"2024-04-26T21:31:21.845313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:39.861976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842264+0000","last_change":"2024-04-26T21:30:52.792458+0000","last_active":"2024-04-26T21:31:21.842264+0000","last_peered":"2024-04-26T21:31:21.842264+0000","last_clean":"2024-04-26T21:31:21.842264+0000","last_became_active":"2024-04-26T21:30:52.792087+0000","last_became_peered":"2024-04-26T21:30:52.792087+0000","last_unstale":"2024-04-26T21:31:21.842264+0000","last_undegraded":"2024-04-26T21:31:21.842264+0000","last_fullsized":"2024-04-26T21:31:21.842264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:59:46.420509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837340+0000","last_change":"2024-04-26T21:30:53.826028+0000","last_active":"2024-04-26T21:31:21.837340+0000","last_peered":"2024-04-26T21:31:21.837340+0000","last_clean":"2024-04-26T21:31:21.837340+0000","last_became_active":"2024-04-26T21:30:53.822405+0000","last_became_peered":"2024-04-26T21:30:53.822405+0000","last_unstale":"2024-04-26T21:31:21.837340+0000","last_undegraded":"2024-04-26T21:31:21.837340+0000","last_fullsized":"2024-04-26T21:31:21.837340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:47.501074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.862987+0000","last_change":"2024-04-26T21:31:21.861728+0000","last_active":"2024-04-26T21:31:21.862987+0000","last_peered":"2024-04-26T21:31:21.862987+0000","last_clean":"2024-04-26T21:31:21.862987+0000","last_became_active":"2024-04-26T21:31:21.850330+0000","last_became_peered":"2024-04-26T21:31:21.850330+0000","last_unstale":"2024-04-26T21:31:21.862987+0000","last_undegraded":"2024-04-26T21:31:21.862987+0000","last_fullsized":"2024-04-26T21:31:21.862987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:05.532071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842726+0000","last_change":"2024-04-26T21:30:53.826016+0000","last_active":"2024-04-26T21:31:21.842726+0000","last_peered":"2024-04-26T21:31:21.842726+0000","last_clean":"2024-04-26T21:31:21.842726+0000","last_became_active":"2024-04-26T21:30:53.824050+0000","last_became_peered":"2024-04-26T21:30:53.824050+0000","last_unstale":"2024-04-26T21:31:21.842726+0000","last_undegraded":"2024-04-26T21:31:21.842726+0000","last_fullsized":"2024-04-26T21:31:21.842726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:35.816936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.847822+0000","last_change":"2024-04-26T21:30:52.811142+0000","last_active":"2024-04-26T21:31:21.847822+0000","last_peered":"2024-04-26T21:31:21.847822+0000","last_clean":"2024-04-26T21:31:21.847822+0000","last_became_active":"2024-04-26T21:30:52.810732+0000","last_became_peered":"2024-04-26T21:30:52.810732+0000","last_unstale":"2024-04-26T21:31:21.847822+0000","last_undegraded":"2024-04-26T21:31:21.847822+0000","last_fullsized":"2024-04-26T21:31:21.847822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:21.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.519829+0000","last_change":"2024-04-26T21:30:53.832000+0000","last_active":"2024-04-26T21:31:22.519829+0000","last_peered":"2024-04-26T21:31:22.519829+0000","last_clean":"2024-04-26T21:31:22.519829+0000","last_became_active":"2024-04-26T21:30:53.831619+0000","last_became_peered":"2024-04-26T21:30:53.831619+0000","last_unstale":"2024-04-26T21:31:22.519829+0000","last_undegraded":"2024-04-26T21:31:22.519829+0000","last_fullsized":"2024-04-26T21:31:22.519829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:11.656424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":108,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:32:19.944458+0000","last_change":"2024-04-26T21:30:52.810311+0000","last_active":"2024-04-26T21:32:19.944458+0000","last_peered":"2024-04-26T21:32:19.944458+0000","last_clean":"2024-04-26T21:32:19.944458+0000","last_became_active":"2024-04-26T21:30:52.809976+0000","last_became_peered":"2024-04-26T21:30:52.809976+0000","last_unstale":"2024-04-26T21:32:19.944458+0000","last_undegraded":"2024-04-26T21:32:19.944458+0000","last_fullsized":"2024-04-26T21:32:19.944458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:49.146730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520562+0000","last_change":"2024-04-26T21:30:53.830315+0000","last_active":"2024-04-26T21:31:22.520562+0000","last_peered":"2024-04-26T21:31:22.520562+0000","last_clean":"2024-04-26T21:31:22.520562+0000","last_became_active":"2024-04-26T21:30:53.829439+0000","last_became_peered":"2024-04-26T21:30:53.829439+0000","last_unstale":"2024-04-26T21:31:22.520562+0000","last_undegraded":"2024-04-26T21:31:22.520562+0000","last_fullsized":"2024-04-26T21:31:22.520562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:49.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.832532+0000","last_change":"2024-04-26T21:30:52.802392+0000","last_active":"2024-04-26T21:31:21.832532+0000","last_peered":"2024-04-26T21:31:21.832532+0000","last_clean":"2024-04-26T21:31:21.832532+0000","last_became_active":"2024-04-26T21:30:52.799978+0000","last_became_peered":"2024-04-26T21:30:52.799978+0000","last_unstale":"2024-04-26T21:31:21.832532+0000","last_undegraded":"2024-04-26T21:31:21.832532+0000","last_fullsized":"2024-04-26T21:31:21.832532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:45.993473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520063+0000","last_change":"2024-04-26T21:30:53.818118+0000","last_active":"2024-04-26T21:31:22.520063+0000","last_peered":"2024-04-26T21:31:22.520063+0000","last_clean":"2024-04-26T21:31:22.520063+0000","last_became_active":"2024-04-26T21:30:53.817712+0000","last_became_peered":"2024-04-26T21:30:53.817712+0000","last_unstale":"2024-04-26T21:31:22.520063+0000","last_undegraded":"2024-04-26T21:31:22.520063+0000","last_fullsized":"2024-04-26T21:31:22.520063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.819569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.828161+0000","last_change":"2024-04-26T21:30:52.807308+0000","last_active":"2024-04-26T21:31:21.828161+0000","last_peered":"2024-04-26T21:31:21.828161+0000","last_clean":"2024-04-26T21:31:21.828161+0000","last_became_active":"2024-04-26T21:30:52.806935+0000","last_became_peered":"2024-04-26T21:30:52.806935+0000","last_unstale":"2024-04-26T21:31:21.828161+0000","last_undegraded":"2024-04-26T21:31:21.828161+0000","last_fullsized":"2024-04-26T21:31:21.828161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:51.754520+0000","last_clean_scrub_stamp":"2024-04-26T21:30:51.754520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:42.549564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.520636+0000","last_change":"2024-04-26T21:30:53.812940+0000","last_active":"2024-04-26T21:31:22.520636+0000","last_peered":"2024-04-26T21:31:22.520636+0000","last_clean":"2024-04-26T21:31:22.520636+0000","last_became_active":"2024-04-26T21:30:53.812492+0000","last_became_peered":"2024-04-26T21:30:53.812492+0000","last_unstale":"2024-04-26T21:31:22.520636+0000","last_undegraded":"2024-04-26T21:31:22.520636+0000","last_fullsized":"2024-04-26T21:31:22.520636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:07.531226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837769+0000","last_change":"2024-04-26T21:30:53.822943+0000","last_active":"2024-04-26T21:31:21.837769+0000","last_peered":"2024-04-26T21:31:21.837769+0000","last_clean":"2024-04-26T21:31:21.837769+0000","last_became_active":"2024-04-26T21:30:53.822443+0000","last_became_peered":"2024-04-26T21:30:53.822443+0000","last_unstale":"2024-04-26T21:31:21.837769+0000","last_undegraded":"2024-04-26T21:31:21.837769+0000","last_fullsized":"2024-04-26T21:31:21.837769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:57.827470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844894+0000","last_change":"2024-04-26T21:30:53.832417+0000","last_active":"2024-04-26T21:31:21.844894+0000","last_peered":"2024-04-26T21:31:21.844894+0000","last_clean":"2024-04-26T21:31:21.844894+0000","last_became_active":"2024-04-26T21:30:53.832001+0000","last_became_peered":"2024-04-26T21:30:53.832001+0000","last_unstale":"2024-04-26T21:31:21.844894+0000","last_undegraded":"2024-04-26T21:31:21.844894+0000","last_fullsized":"2024-04-26T21:31:21.844894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:02.471766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844628+0000","last_change":"2024-04-26T21:30:53.817472+0000","last_active":"2024-04-26T21:31:21.844628+0000","last_peered":"2024-04-26T21:31:21.844628+0000","last_clean":"2024-04-26T21:31:21.844628+0000","last_became_active":"2024-04-26T21:30:53.817053+0000","last_became_peered":"2024-04-26T21:30:53.817053+0000","last_unstale":"2024-04-26T21:31:21.844628+0000","last_undegraded":"2024-04-26T21:31:21.844628+0000","last_fullsized":"2024-04-26T21:31:21.844628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:19.219132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.844506+0000","last_change":"2024-04-26T21:30:53.826748+0000","last_active":"2024-04-26T21:31:21.844506+0000","last_peered":"2024-04-26T21:31:21.844506+0000","last_clean":"2024-04-26T21:31:21.844506+0000","last_became_active":"2024-04-26T21:30:53.826040+0000","last_became_peered":"2024-04-26T21:30:53.826040+0000","last_unstale":"2024-04-26T21:31:21.844506+0000","last_undegraded":"2024-04-26T21:31:21.844506+0000","last_fullsized":"2024-04-26T21:31:21.844506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:18.222414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.037153+0000","last_change":"2024-04-26T21:31:22.037153+0000","last_active":"2024-04-26T21:31:22.037153+0000","last_peered":"2024-04-26T21:31:22.037153+0000","last_clean":"2024-04-26T21:31:22.037153+0000","last_became_active":"2024-04-26T21:31:22.035984+0000","last_became_peered":"2024-04-26T21:31:22.035984+0000","last_unstale":"2024-04-26T21:31:22.037153+0000","last_undegraded":"2024-04-26T21:31:22.037153+0000","last_fullsized":"2024-04-26T21:31:22.037153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:30.880953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.837888+0000","last_change":"2024-04-26T21:30:53.825396+0000","last_active":"2024-04-26T21:31:21.837888+0000","last_peered":"2024-04-26T21:31:21.837888+0000","last_clean":"2024-04-26T21:31:21.837888+0000","last_became_active":"2024-04-26T21:30:53.825042+0000","last_became_peered":"2024-04-26T21:30:53.825042+0000","last_unstale":"2024-04-26T21:31:21.837888+0000","last_undegraded":"2024-04-26T21:31:21.837888+0000","last_fullsized":"2024-04-26T21:31:21.837888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:51:20.064265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.846648+0000","last_change":"2024-04-26T21:30:53.827819+0000","last_active":"2024-04-26T21:31:21.846648+0000","last_peered":"2024-04-26T21:31:21.846648+0000","last_clean":"2024-04-26T21:31:21.846648+0000","last_became_active":"2024-04-26T21:30:53.827027+0000","last_became_peered":"2024-04-26T21:30:53.827027+0000","last_unstale":"2024-04-26T21:31:21.846648+0000","last_undegraded":"2024-04-26T21:31:21.846648+0000","last_fullsized":"2024-04-26T21:31:21.846648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:49.586652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842726+0000","last_change":"2024-04-26T21:30:53.824693+0000","last_active":"2024-04-26T21:31:21.842726+0000","last_peered":"2024-04-26T21:31:21.842726+0000","last_clean":"2024-04-26T21:31:21.842726+0000","last_became_active":"2024-04-26T21:30:53.824278+0000","last_became_peered":"2024-04-26T21:30:53.824278+0000","last_unstale":"2024-04-26T21:31:21.842726+0000","last_undegraded":"2024-04-26T21:31:21.842726+0000","last_fullsized":"2024-04-26T21:31:21.842726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:42.799139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831974+0000","last_change":"2024-04-26T21:30:53.823886+0000","last_active":"2024-04-26T21:31:21.831974+0000","last_peered":"2024-04-26T21:31:21.831974+0000","last_clean":"2024-04-26T21:31:21.831974+0000","last_became_active":"2024-04-26T21:30:53.823443+0000","last_became_peered":"2024-04-26T21:30:53.823443+0000","last_unstale":"2024-04-26T21:31:21.831974+0000","last_undegraded":"2024-04-26T21:31:21.831974+0000","last_fullsized":"2024-04-26T21:31:21.831974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.022073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.842355+0000","last_change":"2024-04-26T21:30:53.825329+0000","last_active":"2024-04-26T21:31:21.842355+0000","last_peered":"2024-04-26T21:31:21.842355+0000","last_clean":"2024-04-26T21:31:21.842355+0000","last_became_active":"2024-04-26T21:30:53.822833+0000","last_became_peered":"2024-04-26T21:30:53.822833+0000","last_unstale":"2024-04-26T21:31:21.842355+0000","last_undegraded":"2024-04-26T21:31:21.842355+0000","last_fullsized":"2024-04-26T21:31:21.842355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.840177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.831228+0000","last_change":"2024-04-26T21:30:53.837657+0000","last_active":"2024-04-26T21:31:21.831228+0000","last_peered":"2024-04-26T21:31:21.831228+0000","last_clean":"2024-04-26T21:31:21.831228+0000","last_became_active":"2024-04-26T21:30:53.837354+0000","last_became_peered":"2024-04-26T21:30:53.837354+0000","last_unstale":"2024-04-26T21:31:21.831228+0000","last_undegraded":"2024-04-26T21:31:21.831228+0000","last_fullsized":"2024-04-26T21:31:21.831228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:34.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:21.827994+0000","last_change":"2024-04-26T21:30:53.811372+0000","last_active":"2024-04-26T21:31:21.827994+0000","last_peered":"2024-04-26T21:31:21.827994+0000","last_clean":"2024-04-26T21:31:21.827994+0000","last_became_active":"2024-04-26T21:30:53.810124+0000","last_became_peered":"2024-04-26T21:30:53.810124+0000","last_unstale":"2024-04-26T21:31:21.827994+0000","last_undegraded":"2024-04-26T21:31:21.827994+0000","last_fullsized":"2024-04-26T21:31:21.827994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:14.134041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:31:22.037154+0000","last_change":"2024-04-26T21:31:22.037154+0000","last_active":"2024-04-26T21:31:22.037154+0000","last_peered":"2024-04-26T21:31:22.037154+0000","last_clean":"2024-04-26T21:31:22.037154+0000","last_became_active":"2024-04-26T21:31:22.035817+0000","last_became_peered":"2024-04-26T21:31:22.035817+0000","last_unstale":"2024-04-26T21:31:22.037154+0000","last_undegraded":"2024-04-26T21:31:22.037154+0000","last_fullsized":"2024-04-26T21:31:22.037154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:30:52.758930+0000","last_clean_scrub_stamp":"2024-04-26T21:30:52.758930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:53.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59839,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":844,"num_read_kb":489,"num_write":236,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":179517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":470384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1113545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":469811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972408,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":470077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1063098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":469528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":522685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":471258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":523575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":469353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":469892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36920,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708232,"statfs":{"total":95995035648,"available":95957229568,"internally_reserved":0,"allocated":1613824,"data_stored":1062385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:32:27.199 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T21:32:27.199 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:32:27.199 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:32:27.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T21:32:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:28.015 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:27 smithi097 ceph-mon[28127]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:28.015 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:27 smithi097 ceph-mon[28127]: pgmap v413: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:28.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:27 smithi143 ceph-mon[31461]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:28.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:27 smithi143 ceph-mon[31461]: pgmap v413: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:27 smithi111 ceph-mon[31001]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:32:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:27 smithi111 ceph-mon[31001]: pgmap v413: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 -- 172.21.15.97:0/3764626635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04075f70 msgr2=0x7ffa0410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 --2- 172.21.15.97:0/3764626635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04075f70 0x7ffa0410f8e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009990 tx=0x7ff9f0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 -- 172.21.15.97:0/3764626635 shutdown_connections 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 --2- 172.21.15.97:0/3764626635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 0x7ffa04075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 --2- 172.21.15.97:0/3764626635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa0410fe20 0x7ffa04112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 --2- 172.21.15.97:0/3764626635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04075f70 0x7ffa0410f8e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 -- 172.21.15.97:0/3764626635 >> 172.21.15.97:0/3764626635 conn(0x7ffa0407a580 msgr2=0x7ffa0407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 -- 172.21.15.97:0/3764626635 shutdown_connections 2024-04-26T21:32:29.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.907+0000 7ffa0b000700 1 -- 172.21.15.97:0/3764626635 wait complete. 2024-04-26T21:32:29.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.908+0000 7ffa0b000700 1 Processor -- start 2024-04-26T21:32:29.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.908+0000 7ffa0b000700 1 -- start start 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 0x7ffa0419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa04075f70 0x7ffa0419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041157d0 con 0x7ffa0410fe20 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04115650 con 0x7ffa04075650 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa0b000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04115950 con 0x7ffa04075f70 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa09000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa09000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59714/0 (socket says 172.21.15.97:59714) 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa09000700 1 -- 172.21.15.97:0/2087285619 learned_addr learned my addr 172.21.15.97:0/2087285619 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa03e00700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 0x7ffa0419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.909+0000 7ffa03400700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa04075f70 0x7ffa0419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 -- 172.21.15.97:0/2087285619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa04075f70 msgr2=0x7ffa0419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa04075f70 0x7ffa0419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 -- 172.21.15.97:0/2087285619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 msgr2=0x7ffa0419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 0x7ffa0419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 -- 172.21.15.97:0/2087285619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa041a0da0 con 0x7ffa0410fe20 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa09000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b9e0 tx=0x7ff9fc00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00d8e0 con 0x7ffa0410fe20 2024-04-26T21:32:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa041a1090 con 0x7ffa0410fe20 2024-04-26T21:32:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9fc015420 con 0x7ffa0410fe20 2024-04-26T21:32:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.910+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc014630 con 0x7ffa0410fe20 2024-04-26T21:32:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.911+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041a8910 con 0x7ffa0410fe20 2024-04-26T21:32:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.912+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0406a110 con 0x7ffa0410fe20 2024-04-26T21:32:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.914+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff9fc0155d0 con 0x7ffa0410fe20 2024-04-26T21:32:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.915+0000 7ffa00c00700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 0x7ff9e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.915+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff9fc0d0380 con 0x7ffa0410fe20 2024-04-26T21:32:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.915+0000 7ffa03e00700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 0x7ff9e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:29.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.916+0000 7ffa03e00700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 0x7ff9e80a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009a10 tx=0x7ff9f4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:29.917+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff9fc09cc10 con 0x7ffa0410fe20 2024-04-26T21:32:30.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:29 smithi143 ceph-mon[31461]: pgmap v414: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:30.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:29 smithi097 ceph-mon[28127]: pgmap v414: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:29 smithi111 ceph-mon[31001]: pgmap v414: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:30.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.186+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffa0402e490 con 0x7ffa0410fe20 2024-04-26T21:32:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.187+0000 7ffa00c00700 1 -- 172.21.15.97:0/2087285619 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff9fc09c510 con 0x7ffa0410fe20 2024-04-26T21:32:30.188 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:30.188 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.189+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 msgr2=0x7ff9e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.189+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 0x7ff9e80a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009a10 tx=0x7ff9f4009190 comp rx=0 tx=0).stop 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 msgr2=0x7ffa041a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b9e0 tx=0x7ff9fc00bcf0 comp rx=0 tx=0).stop 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 shutdown_connections 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff9e80a3cd0 0x7ff9e80a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa04075650 0x7ffa0419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0410fe20 0x7ffa041a0660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 --2- 172.21.15.97:0/2087285619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa04075f70 0x7ffa0419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 >> 172.21.15.97:0/2087285619 conn(0x7ffa0407a580 msgr2=0x7ffa04100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 shutdown_connections 2024-04-26T21:32:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:30.190+0000 7ffa0b000700 1 -- 172.21.15.97:0/2087285619 wait complete. 2024-04-26T21:32:31.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:30 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2087285619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:32:31.015 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:32:31.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:32:31.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:30 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2087285619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:32:31.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:30 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2087285619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:32:31.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 -- 172.21.15.97:0/3134834448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 msgr2=0x7fc140075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 --2- 172.21.15.97:0/3134834448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc140075790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802a480 comp rx=0 tx=0).stop 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 -- 172.21.15.97:0/3134834448 shutdown_connections 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 --2- 172.21.15.97:0/3134834448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc14010a7e0 0x7fc14010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 --2- 172.21.15.97:0/3134834448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc140075790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 --2- 172.21.15.97:0/3134834448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140076910 0x7fc140074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 -- 172.21.15.97:0/3134834448 >> 172.21.15.97:0/3134834448 conn(0x7fc1400fdee0 msgr2=0x7fc140100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.228+0000 7fc146a00640 1 -- 172.21.15.97:0/3134834448 shutdown_connections 2024-04-26T21:32:31.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.229+0000 7fc146a00640 1 -- 172.21.15.97:0/3134834448 wait complete. 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.229+0000 7fc146a00640 1 Processor -- start 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 -- start start 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc140076910 0x7fc14019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc14010a7e0 0x7fc1401a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc140110580 con 0x7fc140075330 2024-04-26T21:32:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc140110400 con 0x7fc14010a7e0 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc146a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc140110700 con 0x7fc140076910 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37904/0 (socket says 172.21.15.97:37904) 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 -- 172.21.15.97:0/1881392784 learned_addr learned my addr 172.21.15.97:0/1881392784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 -- 172.21.15.97:0/1881392784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc140076910 msgr2=0x7fc14019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc140076910 0x7fc14019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 -- 172.21.15.97:0/1881392784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc14010a7e0 msgr2=0x7fc1401a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc14010a7e0 0x7fc1401a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.230+0000 7fc13fe00640 1 -- 172.21.15.97:0/1881392784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401a28e0 con 0x7fc140075330 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc13fe00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc13000d950 tx=0x7fc13000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130017070 con 0x7fc140075330 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc130009c30 con 0x7fc140075330 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401a2bd0 con 0x7fc140075330 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13000f620 con 0x7fc140075330 2024-04-26T21:32:31.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.231+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a3170 con 0x7fc140075330 2024-04-26T21:32:31.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.232+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1401095a0 con 0x7fc140075330 2024-04-26T21:32:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.234+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc130010460 con 0x7fc140075330 2024-04-26T21:32:31.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.236+0000 7fc13d600640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 0x7fc1240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.236+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fc1300d01c0 con 0x7fc140075330 2024-04-26T21:32:31.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.236+0000 7fc137400640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 0x7fc1240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.237+0000 7fc137400640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 0x7fc1240a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc128009230 tx=0x7fc128000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.240+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc13009ca50 con 0x7fc140075330 2024-04-26T21:32:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.445+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc14002dd00 con 0x7fc140075330 2024-04-26T21:32:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.447+0000 7fc13d600640 1 -- 172.21.15.97:0/1881392784 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7fc130009de0 con 0x7fc140075330 2024-04-26T21:32:31.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 msgr2=0x7fc1240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 0x7fc1240a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc128009230 tx=0x7fc128000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 msgr2=0x7fc14019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc13000d950 tx=0x7fc13000de10 comp rx=0 tx=0).stop 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 shutdown_connections 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc14010a7e0 0x7fc1401a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc140076910 0x7fc14019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fc1240a3cc0 0x7fc1240a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 --2- 172.21.15.97:0/1881392784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc140075330 0x7fc14019aca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 >> 172.21.15.97:0/1881392784 conn(0x7fc1400fdee0 msgr2=0x7fc140106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 shutdown_connections 2024-04-26T21:32:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.450+0000 7fc146a00640 1 -- 172.21.15.97:0/1881392784 wait complete. 2024-04-26T21:32:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:32:31.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:32:31.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 -- 172.21.15.97:0/2079280050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc41043f0 msgr2=0x7f1cc410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/2079280050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc41043f0 0x7f1cc410ac80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009ca0 tx=0x7f1cbc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 -- 172.21.15.97:0/2079280050 shutdown_connections 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/2079280050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc41043f0 0x7f1cc410ac80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/2079280050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4103a30 0x7f1cc4103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/2079280050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 -- 172.21.15.97:0/2079280050 >> 172.21.15.97:0/2079280050 conn(0x7f1cc40fddd0 msgr2=0x7f1cc41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 -- 172.21.15.97:0/2079280050 shutdown_connections 2024-04-26T21:32:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.677+0000 7f1cc3e00640 1 -- 172.21.15.97:0/2079280050 wait complete. 2024-04-26T21:32:31.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 Processor -- start 2024-04-26T21:32:31.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 -- start start 2024-04-26T21:32:31.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 0x7f1cc41a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410e250 con 0x7f1cc4103a30 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410e0d0 con 0x7f1cc41043f0 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.678+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410e3d0 con 0x7f1cc4102840 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37930/0 (socket says 172.21.15.97:37930) 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 -- 172.21.15.97:0/1771067448 learned_addr learned my addr 172.21.15.97:0/1771067448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2a00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 -- 172.21.15.97:0/1771067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 msgr2=0x7f1cc419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc3400640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 0x7f1cc41a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 -- 172.21.15.97:0/1771067448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 msgr2=0x7f1cc41a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 0x7f1cc41a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 -- 172.21.15.97:0/1771067448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc41a2750 con 0x7f1cc4103a30 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2a00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc3400640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 0x7f1cc41a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.679+0000 7f1cc2000640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d950 tx=0x7f1cb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.680+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0017070 con 0x7f1cc4103a30 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.680+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cb0009c30 con 0x7f1cc4103a30 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.680+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000f620 con 0x7f1cc4103a30 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.680+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc41a2a40 con 0x7f1cc4103a30 2024-04-26T21:32:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.680+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a2f40 con 0x7f1cc4103a30 2024-04-26T21:32:31.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.681+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc41a31d0 con 0x7f1cc4103a30 2024-04-26T21:32:31.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.683+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1cb0010460 con 0x7f1cc4103a30 2024-04-26T21:32:31.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.684+0000 7f1cbb400640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 0x7f1ca80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:31.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.685+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f1cb00d01c0 con 0x7f1cc4103a30 2024-04-26T21:32:31.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.685+0000 7f1cc2a00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 0x7f1ca80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:31.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.685+0000 7f1cc2a00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 0x7f1ca80a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4103890 tx=0x7f1cb4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.689+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1cb009ca50 con 0x7f1cc4103a30 2024-04-26T21:32:31.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.892+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cc41a3440 con 0x7f1cc4103a30 2024-04-26T21:32:31.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.893+0000 7f1cbb400640 1 -- 172.21.15.97:0/1771067448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f1cb0009de0 con 0x7f1cc4103a30 2024-04-26T21:32:31.895 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:32:31.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 msgr2=0x7f1ca80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 0x7f1ca80a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4103890 tx=0x7f1cb4005d30 comp rx=0 tx=0).stop 2024-04-26T21:32:31.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 msgr2=0x7f1cc419b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:31.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d950 tx=0x7f1cb000de10 comp rx=0 tx=0).stop 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 shutdown_connections 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc41043f0 0x7f1cc41a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1ca80a3cc0 0x7f1ca80a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc4103a30 0x7f1cc419b070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 --2- 172.21.15.97:0/1771067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4102840 0x7f1cc419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.896+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 >> 172.21.15.97:0/1771067448 conn(0x7f1cc40fddd0 msgr2=0x7f1cc4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.897+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 shutdown_connections 2024-04-26T21:32:31.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:31.897+0000 7f1cc3e00640 1 -- 172.21.15.97:0/1771067448 wait complete. 2024-04-26T21:32:31.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:32:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:31 smithi143 ceph-mon[31461]: pgmap v415: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:32.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:31 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1881392784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:31 smithi097 ceph-mon[28127]: pgmap v415: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:32.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:31 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1881392784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:32.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 -- 172.21.15.97:0/3257785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 msgr2=0x7fe488077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3257785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe488077710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe478009900 tx=0x7fe47802a410 comp rx=0 tx=0).stop 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 -- 172.21.15.97:0/3257785957 shutdown_connections 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3257785957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 0x7fe488079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3257785957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 0x7fe488078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3257785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe488077710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 -- 172.21.15.97:0/3257785957 >> 172.21.15.97:0/3257785957 conn(0x7fe4880fffe0 msgr2=0x7fe488102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 -- 172.21.15.97:0/3257785957 shutdown_connections 2024-04-26T21:32:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.125+0000 7fe48ce00640 1 -- 172.21.15.97:0/3257785957 wait complete. 2024-04-26T21:32:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.126+0000 7fe48ce00640 1 Processor -- start 2024-04-26T21:32:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 -- start start 2024-04-26T21:32:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 0x7fe48819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 0x7fe4881a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe488114860 con 0x7fe4880772f0 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881146e0 con 0x7fe488078ed0 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe48ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881149e0 con 0x7fe488078530 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe487400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe487400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37950/0 (socket says 172.21.15.97:37950) 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.127+0000 7fe487400640 1 -- 172.21.15.97:0/3466416137 learned_addr learned my addr 172.21.15.97:0/3466416137 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe47fe00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 0x7fe48819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487e00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 0x7fe4881a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 -- 172.21.15.97:0/3466416137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 msgr2=0x7fe48819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 0x7fe48819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 -- 172.21.15.97:0/3466416137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 msgr2=0x7fe4881a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 0x7fe4881a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 -- 172.21.15.97:0/3466416137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a6c50 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe487400640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe478009230 tx=0x7fe4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47803f030 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe478003c90 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a6ee0 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478035070 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.128+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a7340 con 0x7fe4880772f0 2024-04-26T21:32:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.129+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4881113e0 con 0x7fe4880772f0 2024-04-26T21:32:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.131+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe478042030 con 0x7fe4880772f0 2024-04-26T21:32:32.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.133+0000 7fe484c00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 0x7fe46c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.133+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fe4780f0d20 con 0x7fe4880772f0 2024-04-26T21:32:32.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.133+0000 7fe47fe00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 0x7fe46c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.134+0000 7fe47fe00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 0x7fe46c0a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe470005fd0 tx=0x7fe470005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.137+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe4780bd670 con 0x7fe4880772f0 2024-04-26T21:32:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:31 smithi111 ceph-mon[31001]: pgmap v415: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:31 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1881392784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:32.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.337+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe48802dd80 con 0x7fe4880772f0 2024-04-26T21:32:32.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.338+0000 7fe484c00640 1 -- 172.21.15.97:0/3466416137 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe4780bcf70 con 0x7fe4880772f0 2024-04-26T21:32:32.339 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 msgr2=0x7fe46c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 0x7fe46c0a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe470005fd0 tx=0x7fe470005850 comp rx=0 tx=0).stop 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 msgr2=0x7fe48819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe478009230 tx=0x7fe4780042f0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 shutdown_connections 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe488078ed0 0x7fe4881a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe488078530 0x7fe48819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe46c0a3cc0 0x7fe46c0a6180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 --2- 172.21.15.97:0/3466416137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4880772f0 0x7fe48819f080 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 >> 172.21.15.97:0/3466416137 conn(0x7fe4880fffe0 msgr2=0x7fe488103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 shutdown_connections 2024-04-26T21:32:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.340+0000 7fe48ce00640 1 -- 172.21.15.97:0/3466416137 wait complete. 2024-04-26T21:32:32.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:32:32.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 -- 172.21.15.97:0/934392459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e01031d0 msgr2=0x7f78e01055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 --2- 172.21.15.97:0/934392459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e01031d0 0x7f78e01055c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009ca0 tx=0x7f78d002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 -- 172.21.15.97:0/934392459 shutdown_connections 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 --2- 172.21.15.97:0/934392459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e0108430 0x7f78e010a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 --2- 172.21.15.97:0/934392459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0105b00 0x7f78e0107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 --2- 172.21.15.97:0/934392459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e01031d0 0x7f78e01055c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 -- 172.21.15.97:0/934392459 >> 172.21.15.97:0/934392459 conn(0x7f78e00fcc00 msgr2=0x7f78e00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 -- 172.21.15.97:0/934392459 shutdown_connections 2024-04-26T21:32:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.565+0000 7f78e4c00640 1 -- 172.21.15.97:0/934392459 wait complete. 2024-04-26T21:32:32.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.566+0000 7f78e4c00640 1 Processor -- start 2024-04-26T21:32:32.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.566+0000 7f78e4c00640 1 -- start start 2024-04-26T21:32:32.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e01031d0 0x7f78e019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 0x7f78e01a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010e1b0 con 0x7f78e0105b00 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010e030 con 0x7f78e01031d0 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78e4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010e330 con 0x7f78e0108430 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37960/0 (socket says 172.21.15.97:37960) 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 -- 172.21.15.97:0/512940387 learned_addr learned my addr 172.21.15.97:0/512940387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78df400640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e01031d0 0x7f78e019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 -- 172.21.15.97:0/512940387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 msgr2=0x7f78e01a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dfe00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 0x7f78e01a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 0x7f78e01a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.567+0000 7f78dea00640 1 -- 172.21.15.97:0/512940387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e01031d0 msgr2=0x7f78e019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78dea00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e01031d0 0x7f78e019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78dea00640 1 -- 172.21.15.97:0/512940387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01a26f0 con 0x7f78e0105b00 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78dfe00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 0x7f78e01a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78dea00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d740 tx=0x7f78c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8017070 con 0x7f78e0105b00 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78c800de50 con 0x7f78e0105b00 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01a29e0 con 0x7f78e0105b00 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8004c60 con 0x7f78e0105b00 2024-04-26T21:32:32.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.568+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a2f30 con 0x7f78e0105b00 2024-04-26T21:32:32.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.569+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e0109f80 con 0x7f78e0105b00 2024-04-26T21:32:32.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.571+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f78c8004e00 con 0x7f78e0105b00 2024-04-26T21:32:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.573+0000 7f78d7e00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 0x7f78c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.574+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f78c80cfe40 con 0x7f78e0105b00 2024-04-26T21:32:32.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.574+0000 7f78df400640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 0x7f78c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:32.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.574+0000 7f78df400640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 0x7f78c40a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f78d00319e0 tx=0x7f78d0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.578+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f78c809c7d0 con 0x7f78e0105b00 2024-04-26T21:32:32.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.780+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78e002dcf0 con 0x7f78e0105b00 2024-04-26T21:32:32.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.781+0000 7f78d7e00640 1 -- 172.21.15.97:0/512940387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f78c809c0d0 con 0x7f78e0105b00 2024-04-26T21:32:32.782 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 msgr2=0x7f78c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 0x7f78c40a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f78d00319e0 tx=0x7f78d0005ef0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 msgr2=0x7f78e019b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d740 tx=0x7f78c800dc00 comp rx=0 tx=0).stop 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 shutdown_connections 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e0108430 0x7f78e01a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f78c40a3cc0 0x7f78c40a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78e0105b00 0x7f78e019b010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 --2- 172.21.15.97:0/512940387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e01031d0 0x7f78e019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 >> 172.21.15.97:0/512940387 conn(0x7f78e00fcc00 msgr2=0x7f78e01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 shutdown_connections 2024-04-26T21:32:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:32.784+0000 7f78e4c00640 1 -- 172.21.15.97:0/512940387 wait complete. 2024-04-26T21:32:32.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:32:33.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 -- 172.21.15.97:0/2945668434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07080772f0 msgr2=0x7f0708077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 --2- 172.21.15.97:0/2945668434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07080772f0 0x7f0708077710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009900 tx=0x7f06f802a410 comp rx=0 tx=0).stop 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 -- 172.21.15.97:0/2945668434 shutdown_connections 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 --2- 172.21.15.97:0/2945668434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0708078ed0 0x7f0708079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 --2- 172.21.15.97:0/2945668434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078530 0x7f0708078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 --2- 172.21.15.97:0/2945668434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07080772f0 0x7f0708077710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 -- 172.21.15.97:0/2945668434 >> 172.21.15.97:0/2945668434 conn(0x7f0708100120 msgr2=0x7f0708102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.005+0000 7f070d600640 1 -- 172.21.15.97:0/2945668434 shutdown_connections 2024-04-26T21:32:33.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.006+0000 7f070d600640 1 -- 172.21.15.97:0/2945668434 wait complete. 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.006+0000 7f070d600640 1 Processor -- start 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 -- start start 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07080772f0 0x7f070819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 0x7f07081a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0708114980 con 0x7f0708078530 2024-04-26T21:32:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0708114800 con 0x7f0708078ed0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f070d600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0708114b00 con 0x7f07080772f0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37988/0 (socket says 172.21.15.97:37988) 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 -- 172.21.15.97:0/3126026228 learned_addr learned my addr 172.21.15.97:0/3126026228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0707400640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 0x7f07081a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706a00640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07080772f0 0x7f070819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 -- 172.21.15.97:0/3126026228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07080772f0 msgr2=0x7f070819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07080772f0 0x7f070819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 -- 172.21.15.97:0/3126026228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 msgr2=0x7f07081a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 0x7f07081a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0706000640 1 -- 172.21.15.97:0/3126026228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07081a6d20 con 0x7f0708078530 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.007+0000 7f0707400640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 0x7f07081a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f0706000640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d770 tx=0x7f06f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4017070 con 0x7f0708078530 2024-04-26T21:32:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06f40054b0 con 0x7f0708078530 2024-04-26T21:32:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081a6fb0 con 0x7f0708078530 2024-04-26T21:32:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4004c60 con 0x7f0708078530 2024-04-26T21:32:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.008+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07081a7480 con 0x7f0708078530 2024-04-26T21:32:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.009+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07081117e0 con 0x7f0708078530 2024-04-26T21:32:33.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.012+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f06f4004e00 con 0x7f0708078530 2024-04-26T21:32:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.013+0000 7f06ff400640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.013+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f06f40d0090 con 0x7f0708078530 2024-04-26T21:32:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.014+0000 7f0706a00640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.015+0000 7f0706a00640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 0x7f06ec0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031460 tx=0x7f06f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.018+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f06f409c920 con 0x7f0708078530 2024-04-26T21:32:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:32 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1771067448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:32 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3466416137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:32 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1771067448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:32 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3466416137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:32 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1771067448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:32 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3466416137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.263+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f070802dd20 con 0x7f0708078530 2024-04-26T21:32:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.265+0000 7f06ff400640 1 -- 172.21.15.97:0/3126026228 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7f06f409c220 con 0x7f0708078530 2024-04-26T21:32:33.266 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:33.267 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:32:33.267 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:32:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 msgr2=0x7f06ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 0x7f06ec0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031460 tx=0x7f06f8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 msgr2=0x7f070819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d770 tx=0x7f06f400dc30 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 shutdown_connections 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0708078ed0 0x7f07081a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0708078530 0x7f070819f640 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.268+0000 7f070d600640 1 --2- 172.21.15.97:0/3126026228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07080772f0 0x7f070819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.269+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 >> 172.21.15.97:0/3126026228 conn(0x7f0708100120 msgr2=0x7f0708101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.269+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 shutdown_connections 2024-04-26T21:32:33.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.269+0000 7f070d600640 1 -- 172.21.15.97:0/3126026228 wait complete. 2024-04-26T21:32:33.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:32:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 -- 172.21.15.97:0/2427596492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8104960 msgr2=0x7f5ad8104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 --2- 172.21.15.97:0/2427596492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8104960 0x7f5ad8104d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009900 tx=0x7f5ac002a410 comp rx=0 tx=0).stop 2024-04-26T21:32:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 -- 172.21.15.97:0/2427596492 shutdown_connections 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 --2- 172.21.15.97:0/2427596492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 0x7f5ad810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 --2- 172.21.15.97:0/2427596492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8105b50 0x7f5ad8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 --2- 172.21.15.97:0/2427596492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8104960 0x7f5ad8104d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 -- 172.21.15.97:0/2427596492 >> 172.21.15.97:0/2427596492 conn(0x7f5ad80ffeb0 msgr2=0x7f5ad8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 -- 172.21.15.97:0/2427596492 shutdown_connections 2024-04-26T21:32:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.494+0000 7f5addc00640 1 -- 172.21.15.97:0/2427596492 wait complete. 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.495+0000 7f5addc00640 1 Processor -- start 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.495+0000 7f5addc00640 1 -- start start 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8104960 0x7f5ad819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 0x7f5ad81a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8110390 con 0x7f5ad8105b50 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8110210 con 0x7f5ad8104960 2024-04-26T21:32:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5addc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8110510 con 0x7f5ad8106510 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37996/0 (socket says 172.21.15.97:37996) 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 -- 172.21.15.97:0/2069230365 learned_addr learned my addr 172.21.15.97:0/2069230365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7e00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8104960 0x7f5ad819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5add200640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 0x7f5ad81a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 -- 172.21.15.97:0/2069230365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 msgr2=0x7f5ad81a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 0x7f5ad81a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 -- 172.21.15.97:0/2069230365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8104960 msgr2=0x7f5ad819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8104960 0x7f5ad819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.496+0000 7f5ad7400640 1 -- 172.21.15.97:0/2069230365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81a26f0 con 0x7f5ad8105b50 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5ad7400640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800db80 tx=0x7f5ac800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8017070 con 0x7f5ad8105b50 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ac8004020 con 0x7f5ad8105b50 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad81a29e0 con 0x7f5ad8105b50 2024-04-26T21:32:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800be60 con 0x7f5ad8105b50 2024-04-26T21:32:33.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.497+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a2f30 con 0x7f5ad8105b50 2024-04-26T21:32:33.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.498+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad810bcf0 con 0x7f5ad8105b50 2024-04-26T21:32:33.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.500+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ac8003b70 con 0x7f5ad8105b50 2024-04-26T21:32:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.502+0000 7f5ad4c00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 0x7f5abc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.502+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5ac80d0690 con 0x7f5ad8105b50 2024-04-26T21:32:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.502+0000 7f5ad7e00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 0x7f5abc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.503+0000 7f5ad7e00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 0x7f5abc0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00057c0 tx=0x7f5ac0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.506+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ac80212a0 con 0x7f5ad8105b50 2024-04-26T21:32:33.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.699+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ad802dcf0 con 0x7f5ad8105b50 2024-04-26T21:32:33.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.702+0000 7f5ad4c00640 1 -- 172.21.15.97:0/2069230365 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f5ac809d020 con 0x7f5ad8105b50 2024-04-26T21:32:33.704 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:33.704 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:32:33.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 msgr2=0x7f5abc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 0x7f5abc0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00057c0 tx=0x7f5ac0000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 msgr2=0x7f5ad819b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800db80 tx=0x7f5ac800b630 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 shutdown_connections 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad8106510 0x7f5ad81a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5abc0a3c40 0x7f5abc0a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad8105b50 0x7f5ad819b010 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.704+0000 7f5addc00640 1 --2- 172.21.15.97:0/2069230365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ad8104960 0x7f5ad819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.705+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 >> 172.21.15.97:0/2069230365 conn(0x7f5ad80ffeb0 msgr2=0x7f5ad8108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.705+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 shutdown_connections 2024-04-26T21:32:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.705+0000 7f5addc00640 1 -- 172.21.15.97:0/2069230365 wait complete. 2024-04-26T21:32:33.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:32:33.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:32:33.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 -- 172.21.15.97:0/2207299810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298109fc0 msgr2=0x7f529810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 --2- 172.21.15.97:0/2207299810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298109fc0 0x7f529810a440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5280009900 tx=0x7f528002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 -- 172.21.15.97:0/2207299810 shutdown_connections 2024-04-26T21:32:33.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 --2- 172.21.15.97:0/2207299810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f529810a980 0x7f52981112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 --2- 172.21.15.97:0/2207299810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298109fc0 0x7f529810a440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 --2- 172.21.15.97:0/2207299810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5298108e70 0x7f5298109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 -- 172.21.15.97:0/2207299810 >> 172.21.15.97:0/2207299810 conn(0x7f5298077e00 msgr2=0x7f529807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:33.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 -- 172.21.15.97:0/2207299810 shutdown_connections 2024-04-26T21:32:33.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.921+0000 7f5297e00640 1 -- 172.21.15.97:0/2207299810 wait complete. 2024-04-26T21:32:33.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 Processor -- start 2024-04-26T21:32:33.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 -- start start 2024-04-26T21:32:33.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5298109fc0 0x7f529819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 0x7f52981a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5298114950 con 0x7f5298108e70 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f52981147d0 con 0x7f529810a980 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.922+0000 7f5297e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5298114ad0 con 0x7f5298109fc0 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38012/0 (socket says 172.21.15.97:38012) 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 -- 172.21.15.97:0/1154511091 learned_addr learned my addr 172.21.15.97:0/1154511091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f528fe00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5298109fc0 0x7f529819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 -- 172.21.15.97:0/1154511091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5298109fc0 msgr2=0x7f529819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5298109fc0 0x7f529819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5297400640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 0x7f52981a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 -- 172.21.15.97:0/1154511091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 msgr2=0x7f52981a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:33.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 0x7f52981a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 -- 172.21.15.97:0/1154511091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52981a6d40 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5297400640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 0x7f52981a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.923+0000 7f5296a00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f528800b4a0 tx=0x7f528800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.924+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288014070 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.924+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5288010aa0 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.924+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52981a6fd0 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.924+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288017b70 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.924+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52981a7510 con 0x7f5298108e70 2024-04-26T21:32:33.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.925+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52981102f0 con 0x7f5298108e70 2024-04-26T21:32:33.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.927+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5288017070 con 0x7f5298108e70 2024-04-26T21:32:33.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.929+0000 7f528f400640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 0x7f527c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.929+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f52880cfc00 con 0x7f5298108e70 2024-04-26T21:32:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.929+0000 7f528fe00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 0x7f527c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:33.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.930+0000 7f528fe00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 0x7f527c0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f5280000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:33.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:33.933+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f528809c5a0 con 0x7f5298108e70 2024-04-26T21:32:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:33 smithi143 ceph-mon[31461]: pgmap v416: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:32:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/512940387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3126026228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2069230365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:33 smithi097 ceph-mon[28127]: pgmap v416: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:32:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/512940387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3126026228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2069230365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.131+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5298063530 con 0x7f5298108e70 2024-04-26T21:32:34.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.132+0000 7f528f400640 1 -- 172.21.15.97:0/1154511091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f528809bea0 con 0x7f5298108e70 2024-04-26T21:32:34.133 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 msgr2=0x7f527c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 0x7f527c0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f5280000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 msgr2=0x7f529819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f528800b4a0 tx=0x7f528800b960 comp rx=0 tx=0).stop 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 shutdown_connections 2024-04-26T21:32:34.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f529810a980 0x7f52981a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5298109fc0 0x7f529819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f527c0a3c40 0x7f527c0a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 --2- 172.21.15.97:0/1154511091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5298108e70 0x7f529819f120 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 >> 172.21.15.97:0/1154511091 conn(0x7f5298077e00 msgr2=0x7f529810d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.135+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 shutdown_connections 2024-04-26T21:32:34.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.136+0000 7f5297e00640 1 -- 172.21.15.97:0/1154511091 wait complete. 2024-04-26T21:32:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:33 smithi111 ceph-mon[31001]: pgmap v416: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:32:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/512940387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3126026228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2069230365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:34.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.356+0000 7f8a13200640 1 -- 172.21.15.97:0/2827001018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 msgr2=0x7f8a141060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.356+0000 7f8a13200640 1 --2- 172.21.15.97:0/2827001018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a141060e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009ca0 tx=0x7f89fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 -- 172.21.15.97:0/2827001018 shutdown_connections 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 --2- 172.21.15.97:0/2827001018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a14106620 0x7f8a1410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 --2- 172.21.15.97:0/2827001018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a141060e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 --2- 172.21.15.97:0/2827001018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14104a70 0x7f8a14104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 -- 172.21.15.97:0/2827001018 >> 172.21.15.97:0/2827001018 conn(0x7f8a140fffe0 msgr2=0x7f8a14102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 -- 172.21.15.97:0/2827001018 shutdown_connections 2024-04-26T21:32:34.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.357+0000 7f8a13200640 1 -- 172.21.15.97:0/2827001018 wait complete. 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 Processor -- start 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 -- start start 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a14104a70 0x7f8a1419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a1419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141104a0 con 0x7f8a14106620 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14110320 con 0x7f8a14104a70 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a13200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14110620 con 0x7f8a14105c60 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.358+0000 7f8a12800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38030/0 (socket says 172.21.15.97:38030) 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 -- 172.21.15.97:0/3566563312 learned_addr learned my addr 172.21.15.97:0/3566563312 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a09400640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a1419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 -- 172.21.15.97:0/3566563312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 msgr2=0x7f8a1419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a1419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 -- 172.21.15.97:0/3566563312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a14104a70 msgr2=0x7f8a1419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a14104a70 0x7f8a1419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 -- 172.21.15.97:0/3566563312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a27c0 con 0x7f8a14106620 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a12800640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e970 tx=0x7f8a0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00ccc0 con 0x7f8a14106620 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c00ce60 con 0x7f8a14106620 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.359+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c010430 con 0x7f8a14106620 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.360+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141a2ab0 con 0x7f8a14106620 2024-04-26T21:32:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.360+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a2fb0 con 0x7f8a14106620 2024-04-26T21:32:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.361+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1410be60 con 0x7f8a14106620 2024-04-26T21:32:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.363+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a0c004ef0 con 0x7f8a14106620 2024-04-26T21:32:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.364+0000 7f8a0b400640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.364+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8a0c0d0aa0 con 0x7f8a14106620 2024-04-26T21:32:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.364+0000 7f8a11e00640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.365+0000 7f8a11e00640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 0x7f89f80a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a14105ac0 tx=0x7f8a04009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.368+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a0c09d330 con 0x7f8a14106620 2024-04-26T21:32:34.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.568+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8a1402dd10 con 0x7f8a14106620 2024-04-26T21:32:34.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.569+0000 7f8a0b400640 1 -- 172.21.15.97:0/3566563312 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f8a0c0055d0 con 0x7f8a14106620 2024-04-26T21:32:34.570 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 msgr2=0x7f89f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 0x7f89f80a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a14105ac0 tx=0x7f8a04009630 comp rx=0 tx=0).stop 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 msgr2=0x7f8a141a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e970 tx=0x7f8a0c00ed70 comp rx=0 tx=0).stop 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 shutdown_connections 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a14106620 0x7f8a141a2150 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a14105c60 0x7f8a1419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 --2- 172.21.15.97:0/3566563312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a14104a70 0x7f8a1419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 >> 172.21.15.97:0/3566563312 conn(0x7f8a140fffe0 msgr2=0x7f8a14109030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 shutdown_connections 2024-04-26T21:32:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.572+0000 7f8a13200640 1 -- 172.21.15.97:0/3566563312 wait complete. 2024-04-26T21:32:34.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 -- 172.21.15.97:0/1848829639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 msgr2=0x7f692c105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 --2- 172.21.15.97:0/1848829639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c105800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f691c009900 tx=0x7f691c02a410 comp rx=0 tx=0).stop 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 -- 172.21.15.97:0/1848829639 shutdown_connections 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 --2- 172.21.15.97:0/1848829639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c108670 0x7f692c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 --2- 172.21.15.97:0/1848829639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c105d40 0x7f692c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 --2- 172.21.15.97:0/1848829639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c105800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 -- 172.21.15.97:0/1848829639 >> 172.21.15.97:0/1848829639 conn(0x7f692c0fce40 msgr2=0x7f692c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 -- 172.21.15.97:0/1848829639 shutdown_connections 2024-04-26T21:32:34.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.802+0000 7f6932600640 1 -- 172.21.15.97:0/1848829639 wait complete. 2024-04-26T21:32:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 Processor -- start 2024-04-26T21:32:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 -- start start 2024-04-26T21:32:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 0x7f692c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c108670 0x7f692c1a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10e3e0 con 0x7f692c103410 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10e260 con 0x7f692c108670 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f6932600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10e560 con 0x7f692c105d40 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f692b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.803+0000 7f692b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38046/0 (socket says 172.21.15.97:38046) 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 -- 172.21.15.97:0/1179207599 learned_addr learned my addr 172.21.15.97:0/1179207599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692aa00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 0x7f692c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692be00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c108670 0x7f692c1a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 -- 172.21.15.97:0/1179207599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 msgr2=0x7f692c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 0x7f692c19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 -- 172.21.15.97:0/1179207599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c108670 msgr2=0x7f692c1a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c108670 0x7f692c1a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 -- 172.21.15.97:0/1179207599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a2930 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692aa00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 0x7f692c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f692b400640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f691c009850 tx=0x7f691c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c03f070 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f691c003e20 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c035030 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.804+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1a2bc0 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.805+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a3060 con 0x7f692c103410 2024-04-26T21:32:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.806+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c10a1c0 con 0x7f692c103410 2024-04-26T21:32:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.808+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f691c042070 con 0x7f692c103410 2024-04-26T21:32:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.809+0000 7f6923e00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 0x7f69100a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.809+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f691c0f1190 con 0x7f692c103410 2024-04-26T21:32:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.809+0000 7f692aa00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 0x7f69100a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.810+0000 7f692aa00640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 0x7f69100a6140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69180058c0 tx=0x7f6918005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:34.813+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f691c0bda20 con 0x7f692c103410 2024-04-26T21:32:35.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.004+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f692c063530 con 0x7f692c103410 2024-04-26T21:32:35.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.005+0000 7f6923e00640 1 -- 172.21.15.97:0/1179207599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f691c0bd320 con 0x7f692c103410 2024-04-26T21:32:35.006 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:32:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 msgr2=0x7f69100a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 0x7f69100a6140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69180058c0 tx=0x7f6918005ec0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 msgr2=0x7f692c19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f691c009850 tx=0x7f691c003c00 comp rx=0 tx=0).stop 2024-04-26T21:32:35.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 shutdown_connections 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f692c108670 0x7f692c1a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f692c105d40 0x7f692c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69100a3c80 0x7f69100a6140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 --2- 172.21.15.97:0/1179207599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c103410 0x7f692c19ad20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.007+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 >> 172.21.15.97:0/1179207599 conn(0x7f692c0fce40 msgr2=0x7f692c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.008+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 shutdown_connections 2024-04-26T21:32:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.008+0000 7f6932600640 1 -- 172.21.15.97:0/1179207599 wait complete. 2024-04-26T21:32:35.024 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:34 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1154511091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.024 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:34 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3566563312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:32:35.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:34 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1154511091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:34 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3566563312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:34 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1154511091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:34 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3566563312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:32:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 -- 172.21.15.97:0/895336148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe98109f80 msgr2=0x7efe9810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 --2- 172.21.15.97:0/895336148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe98109f80 0x7efe9810a400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe84009900 tx=0x7efe8402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 -- 172.21.15.97:0/895336148 shutdown_connections 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 --2- 172.21.15.97:0/895336148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe9810a940 0x7efe981111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 --2- 172.21.15.97:0/895336148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe98109f80 0x7efe9810a400 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 --2- 172.21.15.97:0/895336148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 0x7efe981091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 -- 172.21.15.97:0/895336148 >> 172.21.15.97:0/895336148 conn(0x7efe98077d30 msgr2=0x7efe9807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 -- 172.21.15.97:0/895336148 shutdown_connections 2024-04-26T21:32:35.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.219+0000 7efe9d400640 1 -- 172.21.15.97:0/895336148 wait complete. 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.220+0000 7efe9d400640 1 Processor -- start 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.220+0000 7efe9d400640 1 -- start start 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 0x7efe9819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe98109f80 0x7efe9819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe98114840 con 0x7efe9810a940 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efe981146c0 con 0x7efe98109f80 2024-04-26T21:32:35.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe981149c0 con 0x7efe98108d90 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38056/0 (socket says 172.21.15.97:38056) 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9ca00640 1 -- 172.21.15.97:0/3874862255 learned_addr learned my addr 172.21.15.97:0/3874862255 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe97e00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 0x7efe9819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe97400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe98109f80 0x7efe9819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.221+0000 7efe9ca00640 1 -- 172.21.15.97:0/3874862255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 msgr2=0x7efe9819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9ca00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 0x7efe9819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9ca00640 1 -- 172.21.15.97:0/3874862255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe98109f80 msgr2=0x7efe9819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9ca00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe98109f80 0x7efe9819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9ca00640 1 -- 172.21.15.97:0/3874862255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981a6b60 con 0x7efe9810a940 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9ca00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b970 tx=0x7efe9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90019070 con 0x7efe9810a940 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe90014440 con 0x7efe9810a940 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a6e50 con 0x7efe9810a940 2024-04-26T21:32:35.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe900075d0 con 0x7efe9810a940 2024-04-26T21:32:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.222+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a7340 con 0x7efe9810a940 2024-04-26T21:32:35.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.224+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe98110100 con 0x7efe9810a940 2024-04-26T21:32:35.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.225+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efe9001c070 con 0x7efe9810a940 2024-04-26T21:32:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.227+0000 7efe94c00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 0x7efe7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.227+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7efe900cfa70 con 0x7efe9810a940 2024-04-26T21:32:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.227+0000 7efe97e00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 0x7efe7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.228+0000 7efe97e00640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 0x7efe7c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe88005fd0 tx=0x7efe88005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:35.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.231+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe9009c410 con 0x7efe9810a940 2024-04-26T21:32:35.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.475+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe9802dad0 con 0x7efe9810a940 2024-04-26T21:32:35.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.477+0000 7efe94c00640 1 -- 172.21.15.97:0/3874862255 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7efe9009bd10 con 0x7efe9810a940 2024-04-26T21:32:35.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:35.478 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:32:35.478 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:32:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.479+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 msgr2=0x7efe7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.479+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 0x7efe7c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe88005fd0 tx=0x7efe88005850 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.479+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 msgr2=0x7efe981a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.479+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b970 tx=0x7efe9000be30 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 shutdown_connections 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efe7c0a3c40 0x7efe7c0a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe9810a940 0x7efe981a64f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe98109f80 0x7efe9819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 --2- 172.21.15.97:0/3874862255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe98108d90 0x7efe9819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 >> 172.21.15.97:0/3874862255 conn(0x7efe98077d30 msgr2=0x7efe9810d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 shutdown_connections 2024-04-26T21:32:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.480+0000 7efe9d400640 1 -- 172.21.15.97:0/3874862255 wait complete. 2024-04-26T21:32:35.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:32:35.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 -- 172.21.15.97:0/2262802288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 msgr2=0x7f8238075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 --2- 172.21.15.97:0/2262802288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f8238075810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8224009930 tx=0x7f822402a420 comp rx=0 tx=0).stop 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 -- 172.21.15.97:0/2262802288 shutdown_connections 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 --2- 172.21.15.97:0/2262802288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 0x7f8238111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 --2- 172.21.15.97:0/2262802288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f8238075810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 --2- 172.21.15.97:0/2262802288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 0x7f8238074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 -- 172.21.15.97:0/2262802288 >> 172.21.15.97:0/2262802288 conn(0x7f82380fdee0 msgr2=0x7f8238100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 -- 172.21.15.97:0/2262802288 shutdown_connections 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.697+0000 7f823da00640 1 -- 172.21.15.97:0/2262802288 wait complete. 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 Processor -- start 2024-04-26T21:32:35.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 -- start start 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 0x7f823819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 0x7f82381a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82381149f0 con 0x7f8238075390 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8238114870 con 0x7f8238076970 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f823da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8238114b70 con 0x7f823810ef40 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.698+0000 7f8236a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38076/0 (socket says 172.21.15.97:38076) 2024-04-26T21:32:35.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 -- 172.21.15.97:0/664482677 learned_addr learned my addr 172.21.15.97:0/664482677 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236000640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 0x7f823819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8237400640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 0x7f82381a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 -- 172.21.15.97:0/664482677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 msgr2=0x7f82381a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 0x7f82381a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 -- 172.21.15.97:0/664482677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 msgr2=0x7f823819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 0x7f823819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 -- 172.21.15.97:0/664482677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82381a6d30 con 0x7f8238075390 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f8236a00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f822800d950 tx=0x7f822800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.699+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228017070 con 0x7f8238075390 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.700+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8228009c30 con 0x7f8238075390 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.700+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800f620 con 0x7f8238075390 2024-04-26T21:32:35.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.700+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381a6fc0 con 0x7f8238075390 2024-04-26T21:32:35.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.700+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a74b0 con 0x7f8238075390 2024-04-26T21:32:35.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.701+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8238105110 con 0x7f8238075390 2024-04-26T21:32:35.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.703+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8228010460 con 0x7f8238075390 2024-04-26T21:32:35.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.704+0000 7f822f400640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 0x7f821c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:35.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.704+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f82280d01c0 con 0x7f8238075390 2024-04-26T21:32:35.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.704+0000 7f8236000640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 0x7f821c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:35.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.705+0000 7f8236000640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 0x7f821c0a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82240057c0 tx=0x7f8224000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:35.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.708+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f822809ca50 con 0x7f8238075390 2024-04-26T21:32:35.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.909+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f823802dd60 con 0x7f8238075390 2024-04-26T21:32:35.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.911+0000 7f822f400640 1 -- 172.21.15.97:0/664482677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f8228009de0 con 0x7f8238075390 2024-04-26T21:32:35.912 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:35.912 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.913+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 msgr2=0x7f821c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.913+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 0x7f821c0a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f82240057c0 tx=0x7f8224000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 msgr2=0x7f823819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f822800d950 tx=0x7f822800de10 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 shutdown_connections 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f823810ef40 0x7f82381a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8238076970 0x7f823819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f821c0a3cc0 0x7f821c0a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 --2- 172.21.15.97:0/664482677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8238075390 0x7f823819f110 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 >> 172.21.15.97:0/664482677 conn(0x7f82380fdee0 msgr2=0x7f82381022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 shutdown_connections 2024-04-26T21:32:35.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:35.914+0000 7f823da00640 1 -- 172.21.15.97:0/664482677 wait complete. 2024-04-26T21:32:35.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:32:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:35 smithi097 ceph-mon[28127]: pgmap v417: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1179207599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:36.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3874862255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.127+0000 7f0e94600640 1 -- 172.21.15.97:0/1317702237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 msgr2=0x7f0e8c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.127+0000 7f0e94600640 1 --2- 172.21.15.97:0/1317702237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c10a600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009900 tx=0x7f0e7802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 -- 172.21.15.97:0/1317702237 shutdown_connections 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 --2- 172.21.15.97:0/1317702237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 0x7f0e8c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 --2- 172.21.15.97:0/1317702237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c10a600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 --2- 172.21.15.97:0/1317702237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 0x7f0e8c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 -- 172.21.15.97:0/1317702237 >> 172.21.15.97:0/1317702237 conn(0x7f0e8c077f40 msgr2=0x7f0e8c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 -- 172.21.15.97:0/1317702237 shutdown_connections 2024-04-26T21:32:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.128+0000 7f0e94600640 1 -- 172.21.15.97:0/1317702237 wait complete. 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 Processor -- start 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 -- start start 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 0x7f0e8c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 0x7f0e8c1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114a10 con 0x7f0e8c10a180 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114890 con 0x7f0e8c108f90 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e94600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114b90 con 0x7f0e8c10ab40 2024-04-26T21:32:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38098/0 (socket says 172.21.15.97:38098) 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.129+0000 7f0e91200640 1 -- 172.21.15.97:0/3321764038 learned_addr learned my addr 172.21.15.97:0/3321764038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91c00640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 0x7f0e8c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e92600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 0x7f0e8c1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 -- 172.21.15.97:0/3321764038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 msgr2=0x7f0e8c1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 0x7f0e8c1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 -- 172.21.15.97:0/3321764038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 msgr2=0x7f0e8c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 0x7f0e8c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 -- 172.21.15.97:0/3321764038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c1a6d60 con 0x7f0e8c10a180 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e91200640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e780057c0 tx=0x7f0e78003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7803f070 con 0x7f0e8c10a180 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.130+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e78031da0 con 0x7f0e8c10a180 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.131+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1a6ff0 con 0x7f0e8c10a180 2024-04-26T21:32:36.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.131+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78035030 con 0x7f0e8c10a180 2024-04-26T21:32:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.131+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a74d0 con 0x7f0e8c10a180 2024-04-26T21:32:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.132+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c110250 con 0x7f0e8c10a180 2024-04-26T21:32:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.134+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e78042070 con 0x7f0e8c10a180 2024-04-26T21:32:36.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.135+0000 7f0e8a000640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 0x7f0e740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.135+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f0e780f0fe0 con 0x7f0e8c10a180 2024-04-26T21:32:36.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.135+0000 7f0e91c00640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 0x7f0e740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.136+0000 7f0e91c00640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 0x7f0e740a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e800058c0 tx=0x7f0e80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.140+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0e780bd8d0 con 0x7f0e8c10a180 2024-04-26T21:32:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:35 smithi111 ceph-mon[31001]: pgmap v417: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:35 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1179207599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:35 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3874862255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:36.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:35 smithi143 ceph-mon[31461]: pgmap v417: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:32:36.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:35 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1179207599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:32:36.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:35 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3874862255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.364+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e8c02dad0 con 0x7f0e8c10a180 2024-04-26T21:32:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.366+0000 7f0e8a000640 1 -- 172.21.15.97:0/3321764038 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7f0e780bd1d0 con 0x7f0e8c10a180 2024-04-26T21:32:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:36.367 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:32:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:32:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 msgr2=0x7f0e740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 0x7f0e740a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e800058c0 tx=0x7f0e80005850 comp rx=0 tx=0).stop 2024-04-26T21:32:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 msgr2=0x7f0e8c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e780057c0 tx=0x7f0e78003cd0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 shutdown_connections 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e8c10ab40 0x7f0e8c1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0e740a3c40 0x7f0e740a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e8c10a180 0x7f0e8c19f680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 --2- 172.21.15.97:0/3321764038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8c108f90 0x7f0e8c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.368+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 >> 172.21.15.97:0/3321764038 conn(0x7f0e8c077f40 msgr2=0x7f0e8c10d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.369+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 shutdown_connections 2024-04-26T21:32:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.369+0000 7f0e94600640 1 -- 172.21.15.97:0/3321764038 wait complete. 2024-04-26T21:32:36.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:30:53.245300+0000', 'modified': '2024-04-26T21:31:00.694971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34502, 'mds_1': 34460, 'mds_2': 34433, 'mds_3': 34400, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/3674554302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3674554302}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3674554302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/1900627986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 1900627986}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 1900627986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6833/845773802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 845773802}, {'type': 'v1', 'addr': '172.21.15.143:6833', 'nonce': 845773802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/4086222297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 4086222297}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 4086222297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/1658655529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 1658655529}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 1658655529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.143:6839/2048200369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2048200369}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2048200369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34400': {'gid': 34400, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.111:6833/4057011733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 4057011733}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 4057011733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34433': {'gid': 34433, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6835/738887742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 738887742}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 738887742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.111:6837/1944730061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1944730061}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1944730061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34502': {'gid': 34502, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2198422773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2198422773}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2198422773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34502, 'qdb_cluster': [34400, 24487, 34433, 34502, 34460]} 2024-04-26T21:32:36.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:32:36.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 -- 172.21.15.97:0/848575298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f8078530 msgr2=0x7f90f8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 --2- 172.21.15.97:0/848575298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f8078530 0x7f90f8078990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 -- 172.21.15.97:0/848575298 shutdown_connections 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 --2- 172.21.15.97:0/848575298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f8079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 --2- 172.21.15.97:0/848575298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f8078530 0x7f90f8078990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 --2- 172.21.15.97:0/848575298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f80772f0 0x7f90f8077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 -- 172.21.15.97:0/848575298 >> 172.21.15.97:0/848575298 conn(0x7f90f81000f0 msgr2=0x7f90f8102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 -- 172.21.15.97:0/848575298 shutdown_connections 2024-04-26T21:32:36.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.583+0000 7f90fd600640 1 -- 172.21.15.97:0/848575298 wait complete. 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 Processor -- start 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 -- start start 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f8078530 0x7f90f819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f81a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8114970 con 0x7f90f80772f0 2024-04-26T21:32:36.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81147f0 con 0x7f90f8078ed0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.584+0000 7f90fd600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8114af0 con 0x7f90f8078530 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38118/0 (socket says 172.21.15.97:38118) 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 -- 172.21.15.97:0/1351126172 learned_addr learned my addr 172.21.15.97:0/1351126172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6000640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f8078530 0x7f90f819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f7400640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f81a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 -- 172.21.15.97:0/1351126172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f8078530 msgr2=0x7f90f819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f8078530 0x7f90f819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 -- 172.21.15.97:0/1351126172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 msgr2=0x7f90f81a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f81a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 -- 172.21.15.97:0/1351126172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f81a6c60 con 0x7f90f80772f0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f7400640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f81a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.585+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c920 tx=0x7f90e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.586+0000 7f90ef400640 1 -- 172.21.15.97:0/1351126172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8007c70 con 0x7f90f80772f0 2024-04-26T21:32:36.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.586+0000 7f90ef400640 1 -- 172.21.15.97:0/1351126172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90e8007e10 con 0x7f90f80772f0 2024-04-26T21:32:36.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.586+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f81a6f50 con 0x7f90f80772f0 2024-04-26T21:32:36.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.586+0000 7f90ef400640 1 -- 172.21.15.97:0/1351126172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8018710 con 0x7f90f80772f0 2024-04-26T21:32:36.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.586+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f81a73c0 con 0x7f90f80772f0 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 Processor -- start 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 -- start start 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f90ef400640 1 -- 172.21.15.97:0/1351126172 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f90e800f450 con 0x7f90f80772f0 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c9c50 0x561d939cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561d93928600 con 0x561d939c40f0 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d938c5100 con 0x561d939c9c50 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.589+0000 7f91003a7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d939245c0 con 0x561d939c4ae0 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38126/0 (socket says 172.21.15.97:38126) 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 -- 172.21.15.97:0/1446692439 learned_addr learned my addr 172.21.15.97:0/1446692439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 -- 172.21.15.97:0/1446692439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 msgr2=0x561d939c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f6000640 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c9c50 0x561d939cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 -- 172.21.15.97:0/1446692439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c9c50 msgr2=0x561d939cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c9c50 0x561d939cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 -- 172.21.15.97:0/1446692439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9363a2d0 con 0x561d939c40f0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.590+0000 7f90f7400640 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90f0008dd0 tx=0x7f90f000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d2ffb4d1d844db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90ef400640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 0x7f90dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90e3400640 1 -- 172.21.15.97:0/1446692439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000f830 con 0x561d939c40f0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90e3400640 1 -- 172.21.15.97:0/1446692439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90f000de20 con 0x561d939c40f0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90e3400640 1 -- 172.21.15.97:0/1446692439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018ce0 con 0x561d939c40f0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90ef400640 1 -- 172.21.15.97:0/1351126172 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f90e80d0760 con 0x7f90f80772f0 2024-04-26T21:32:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.591+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 0x7f90dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- 172.21.15.97:0/1446692439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 msgr2=0x561d939c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f90f6a00640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 0x7f90dc0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561d939c4ee0 tx=0x7f90e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90f0008dd0 tx=0x7f90f000d710 comp rx=0 tx=0).stop 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- 172.21.15.97:0/1446692439 shutdown_connections 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c9c50 0x561d939cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 --2- 172.21.15.97:0/1446692439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c40f0 0x561d939c4510 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- 172.21.15.97:0/1446692439 >> 172.21.15.97:0/1446692439 conn(0x561d93958d10 msgr2=0x561d939bfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- 172.21.15.97:0/1446692439 shutdown_connections 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- 172.21.15.97:0/1446692439 wait complete. 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.592+0000 7f91003a7740 1 -- start start 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 Processor -- start 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 0x561d939d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561d939ccc00 con 0x561d939c4ae0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d938aff70 con 0x561d939c40f0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d939ccd80 con 0x561d939c9c50 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f91003a7740 10 client.0 _ll_register_callbacks cb 0x561d9391e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f90f6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f90f6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38128/0 (socket says 172.21.15.97:38128) 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.593+0000 7f90f6000640 1 -- 172.21.15.97:0/2097023828 learned_addr learned my addr 172.21.15.97:0/2097023828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f7400640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f7400640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 0x561d939d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 -- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 msgr2=0x561d939d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 0x561d939d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 -- 172.21.15.97:0/2097023828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 msgr2=0x561d939ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 -- 172.21.15.97:0/2097023828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d937046b0 con 0x561d939c4ae0 2024-04-26T21:32:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f7400640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 0x561d939d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90f6000640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f90e402a9e0 tx=0x7f90e40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b661ffaee0bb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4035030 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f90e4003a00 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.594+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f000de20 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90e4003c00 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561d939d8ba0 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90f000de20 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90e403c440 con 0x561d939c4ae0 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 20 client.15081 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d939db240 0x561d939dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 4 client.15081 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:32:36.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.595+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 --> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d93936210 con 0x561d939db240 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.596+0000 7f90f7400640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d939db240 0x561d939dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.597+0000 7f90d7e00640 10 client.15081 ms_handle_connect on v2:172.21.15.143:6838/2048200369 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.597+0000 7f90f7400640 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d939db240 0x561d939dc6b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=712ff4bd421b73a0 server_cookie=2bf89b7eed6fa01d in_seq=0 out_seq=0 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f90d7e00640 1 -- 172.21.15.97:0/2097023828 <== mds.4 v2:172.21.15.143:6838/2048200369 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d93936210 con 0x561d939db240 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f90d7e00640 10 client.15081 handle_command_reply: tid=0 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 client.15081 shutdown 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 msgr2=0x561d939ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f90e402a9e0 tx=0x7f90e40043b0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 shutdown_connections 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d939db240 0x561d939dc6b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c9c50 0x561d939d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c4ae0 0x561d939ceac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 --2- 172.21.15.97:0/2097023828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f90d7e00640 0 client.15081 ms_handle_reset on v2:172.21.15.143:6838/2048200369 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 >> 172.21.15.97:0/2097023828 conn(0x561d93958d10 msgr2=0x561d939beeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 shutdown_connections 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.599+0000 7f91003a7740 1 -- 172.21.15.97:0/2097023828 wait complete. 2024-04-26T21:32:36.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.600+0000 7f91003a7740 20 client.15081 trim_cache size 0 max 16384 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 Processor -- start 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 -- start start 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561d939da9f0 con 0x561d939c9c50 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d939db0c0 con 0x561d939c40f0 2024-04-26T21:32:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f91003a7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d939db790 con 0x561d939c4ae0 2024-04-26T21:32:36.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.620+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38136/0 (socket says 172.21.15.97:38136) 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 -- 172.21.15.97:0/1020534318 learned_addr learned my addr 172.21.15.97:0/1020534318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f6000640 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 -- 172.21.15.97:0/1020534318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 msgr2=0x561d939d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 -- 172.21.15.97:0/1020534318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 msgr2=0x561d939d6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939d6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 -- 172.21.15.97:0/1020534318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d939c5370 con 0x561d939c9c50 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.621+0000 7f90f7400640 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000f80 tx=0x7f90f0018480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b3a8de2d59100a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f90e2a00640 1 -- 172.21.15.97:0/1020534318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018950 con 0x561d939c9c50 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f90e2a00640 1 -- 172.21.15.97:0/1020534318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90f002be50 con 0x561d939c9c50 2024-04-26T21:32:36.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f90e2a00640 1 -- 172.21.15.97:0/1020534318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f002a3a0 con 0x561d939c9c50 2024-04-26T21:32:36.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 -- 172.21.15.97:0/1020534318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 msgr2=0x561d939bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000f80 tx=0x7f90f0018480 comp rx=0 tx=0).stop 2024-04-26T21:32:36.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 -- 172.21.15.97:0/1020534318 shutdown_connections 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939bf5f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c4ae0 0x561d939d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 --2- 172.21.15.97:0/1020534318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c40f0 0x561d939d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 -- 172.21.15.97:0/1020534318 >> 172.21.15.97:0/1020534318 conn(0x561d93958d10 msgr2=0x561d939cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.622+0000 7f91003a7740 1 -- 172.21.15.97:0/1020534318 shutdown_connections 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 -- 172.21.15.97:0/1020534318 wait complete. 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 -- start start 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 Processor -- start 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c40f0 0x561d939d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c4ae0 0x561d939d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561d939dc530 con 0x561d939c9c50 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d939dbe60 con 0x561d939c4ae0 2024-04-26T21:32:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.623+0000 7f91003a7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d939dcc00 con 0x561d939c40f0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f91003a7740 10 client.0 _ll_register_callbacks cb 0x7f90e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38142/0 (socket says 172.21.15.97:38142) 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 -- 172.21.15.97:0/3500068852 learned_addr learned my addr 172.21.15.97:0/3500068852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c40f0 0x561d939d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 -- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c40f0 msgr2=0x561d939d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c40f0 0x561d939d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 -- 172.21.15.97:0/3500068852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c4ae0 msgr2=0x561d939d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c4ae0 0x561d939d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 -- 172.21.15.97:0/3500068852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d939d4e80 con 0x561d939c9c50 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90f7400640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90f000ffd0 tx=0x7f90f0008950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c4ef27f35264b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.624+0000 7f90d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018b50 con 0x561d939c9c50 2024-04-26T21:32:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f90f0018cf0 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f002be50 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90f000f630 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f91003a7740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561d93a41370 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90f002be50 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.625+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90f002a420 con 0x561d939c9c50 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 20 client.15087 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d93a458d0 0x561d93a445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 4 client.15087 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:32:36.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 --> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] -- command(tid 0: {"prefix": "status"}) -- 0x561d93930080 con 0x561d93a458d0 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.626+0000 7f90f6000640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d93a458d0 0x561d93a445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.627+0000 7f90f6000640 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d93a458d0 0x561d93a445e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e54093024fe841de server_cookie=a6549dcce137fa6a in_seq=0 out_seq=0 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.627+0000 7f90d7e00640 10 client.15087 ms_handle_connect on v2:172.21.15.143:6838/2048200369 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f90d7e00640 1 -- 172.21.15.97:0/3500068852 <== mds.4 v2:172.21.15.143:6838/2048200369 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561d93930080 con 0x561d93a458d0 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f90d7e00640 10 client.15087 handle_command_reply: tid=0 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 client.15087 shutdown 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 msgr2=0x561d939d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90f000ffd0 tx=0x7f90f0008950 comp rx=0 tx=0).stop 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 shutdown_connections 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x561d93a458d0 0x561d93a445e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561d939c9c50 0x561d939d4890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d939c4ae0 0x561d939d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 --2- 172.21.15.97:0/3500068852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d939c40f0 0x561d939d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f90d7e00640 0 client.15087 ms_handle_reset on v2:172.21.15.143:6838/2048200369 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 >> 172.21.15.97:0/3500068852 conn(0x561d93958d10 msgr2=0x561d939d6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 shutdown_connections 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 1 -- 172.21.15.97:0/3500068852 wait complete. 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.628+0000 7f91003a7740 20 client.15087 trim_cache size 0 max 16384 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24487, 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:32:36.632 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 96.306259978, 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 121.29129072400001 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:32:36.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 msgr2=0x7f90dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 0x7f90dc0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x561d939c4ee0 tx=0x7f90e8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 msgr2=0x7f90f819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c920 tx=0x7f90e800cde0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 shutdown_connections 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90f8078ed0 0x7f90f81a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f8078530 0x7f90f819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f90dc0a3cc0 0x7f90dc0a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 --2- 172.21.15.97:0/1351126172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90f80772f0 0x7f90f819f040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 >> 172.21.15.97:0/1351126172 conn(0x7f90f81000f0 msgr2=0x7f90f8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 shutdown_connections 2024-04-26T21:32:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.629+0000 7f90fd600640 1 -- 172.21.15.97:0/1351126172 wait complete. 2024-04-26T21:32:36.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 -- 172.21.15.97:0/3475678553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c3410a500 msgr2=0x7f9c340738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 --2- 172.21.15.97:0/3475678553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c3410a500 0x7f9c340738e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009900 tx=0x7f9c2402a410 comp rx=0 tx=0).stop 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 -- 172.21.15.97:0/3475678553 shutdown_connections 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 --2- 172.21.15.97:0/3475678553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 0x7f9c34113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 --2- 172.21.15.97:0/3475678553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c34073e20 0x7f9c34074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.844+0000 7f9c39200640 1 --2- 172.21.15.97:0/3475678553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c3410a500 0x7f9c340738e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.845+0000 7f9c39200640 1 -- 172.21.15.97:0/3475678553 >> 172.21.15.97:0/3475678553 conn(0x7f9c3406f750 msgr2=0x7f9c34071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.845+0000 7f9c39200640 1 -- 172.21.15.97:0/3475678553 shutdown_connections 2024-04-26T21:32:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.845+0000 7f9c39200640 1 -- 172.21.15.97:0/3475678553 wait complete. 2024-04-26T21:32:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.845+0000 7f9c39200640 1 Processor -- start 2024-04-26T21:32:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 -- start start 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 0x7f9c3410ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 0x7f9c3410b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c341169a0 con 0x7f9c34073e20 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9c34116820 con 0x7f9c34110c70 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c39200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c34116b20 con 0x7f9c3410a500 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38162/0 (socket says 172.21.15.97:38162) 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.846+0000 7f9c33400640 1 -- 172.21.15.97:0/3380726227 learned_addr learned my addr 172.21.15.97:0/3380726227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c32a00640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 0x7f9c3410ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33e00640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 0x7f9c3410b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 -- 172.21.15.97:0/3380726227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 msgr2=0x7f9c3410ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 0x7f9c3410ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 -- 172.21.15.97:0/3380726227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 msgr2=0x7f9c3410b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 0x7f9c3410b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 -- 172.21.15.97:0/3380726227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3410b9d0 con 0x7f9c34073e20 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c32a00640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 0x7f9c3410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c33400640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9c24031460 tx=0x7f9c24003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c2be00640 1 -- 172.21.15.97:0/3380726227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2403f070 con 0x7f9c34073e20 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c341ab1b0 con 0x7f9c34073e20 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c2be00640 1 -- 172.21.15.97:0/3380726227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c24031da0 con 0x7f9c34073e20 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.847+0000 7f9c2be00640 1 -- 172.21.15.97:0/3380726227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24035030 con 0x7f9c34073e20 2024-04-26T21:32:36.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.848+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c341ab750 con 0x7f9c34073e20 2024-04-26T21:32:36.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 Processor -- start 2024-04-26T21:32:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 -- start start 2024-04-26T21:32:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c2be00640 1 -- 172.21.15.97:0/3380726227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c24042070 con 0x7f9c34073e20 2024-04-26T21:32:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c65cb0 0x5643f6c6a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c6ae20 0x5643f6c6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643f6bc9600 con 0x5643f6c652c0 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643f6b66100 con 0x5643f6c6ae20 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643f6bc55c0 con 0x5643f6c65cb0 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.852+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38166/0 (socket says 172.21.15.97:38166) 2024-04-26T21:32:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 -- 172.21.15.97:0/3746818438 learned_addr learned my addr 172.21.15.97:0/3746818438 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 -- 172.21.15.97:0/3746818438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c65cb0 msgr2=0x5643f6c6a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33400640 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c65cb0 0x5643f6c6a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c65cb0 0x5643f6c6a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 -- 172.21.15.97:0/3746818438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c6ae20 msgr2=0x5643f6c6d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c32a00640 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c6ae20 0x5643f6c6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c6ae20 0x5643f6c6d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 -- 172.21.15.97:0/3746818438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f68db2d0 con 0x5643f6c652c0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.853+0000 7f9c33e00640 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009eb0 tx=0x7f9c2c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d79bbb2b03def2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c28c00640 1 -- 172.21.15.97:0/3746818438 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c016070 con 0x5643f6c652c0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c28c00640 1 -- 172.21.15.97:0/3746818438 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c2c004d20 con 0x5643f6c652c0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c28c00640 1 -- 172.21.15.97:0/3746818438 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c012690 con 0x5643f6c652c0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c2be00640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 0x7f9c180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c2be00640 1 -- 172.21.15.97:0/3380726227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9c240f0e50 con 0x7f9c34073e20 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.854+0000 7f9c33400640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 0x7f9c180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/3746818438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 msgr2=0x5643f6c656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009eb0 tx=0x7f9c2c010740 comp rx=0 tx=0).stop 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c33400640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 0x7f9c180a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5643f6c66720 tx=0x7f9c24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/3746818438 shutdown_connections 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c6ae20 0x5643f6c6d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c65cb0 0x5643f6c6a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/3746818438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c656e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/3746818438 >> 172.21.15.97:0/3746818438 conn(0x5643f6bf9d10 msgr2=0x5643f6c60d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/3746818438 shutdown_connections 2024-04-26T21:32:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/3746818438 wait complete. 2024-04-26T21:32:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 -- start start 2024-04-26T21:32:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.855+0000 7f9c3d5ee740 1 Processor -- start 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c76cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c6ddd0 con 0x5643f6c652c0 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643f6b50f70 con 0x5643f6c65cb0 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c6df50 con 0x5643f6c6ae20 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c3d5ee740 10 client.0 _ll_register_callbacks cb 0x5643f6bbf480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38180/0 (socket says 172.21.15.97:38180) 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 -- 172.21.15.97:0/658698699 learned_addr learned my addr 172.21.15.97:0/658698699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 -- 172.21.15.97:0/658698699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 msgr2=0x5643f6c76cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c76cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 -- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 msgr2=0x5643f6c6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.856+0000 7f9c33e00640 1 -- 172.21.15.97:0/658698699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f69a56b0 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c33e00640 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66708187aecf78fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c028060 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9c2c005430 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c2c004d20 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c016070 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c3d5ee740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643f6c79ca0 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.857+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c2c004d20 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c2c00f040 con 0x5643f6c652c0 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 10 client.15099 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:32:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 20 client.15099 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6c7bf50 0x5643f6c7d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 4 client.15099 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 --> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f6bd7210 con 0x5643f6c7bf50 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.858+0000 7f9c32a00640 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6c7bf50 0x5643f6c7d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.859+0000 7f9c32a00640 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6c7bf50 0x5643f6c7d3c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be6f21a56a5a3ade server_cookie=febbe5bdc8e47a5a in_seq=0 out_seq=0 2024-04-26T21:32:36.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.859+0000 7f9c1d600640 10 client.15099 ms_handle_connect on v2:172.21.15.111:6832/4057011733 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c1d600640 1 -- 172.21.15.97:0/658698699 <== mds.3 v2:172.21.15.111:6832/4057011733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643f6bd7210 con 0x5643f6c7bf50 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c1d600640 10 client.15099 handle_command_reply: tid=0 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 client.15099 shutdown 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 msgr2=0x5643f6c6f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c005100 comp rx=0 tx=0).stop 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 shutdown_connections 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6c7bf50 0x5643f6c7d3c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c76cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c1d600640 0 client.15099 ms_handle_reset on v2:172.21.15.111:6832/4057011733 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/658698699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c652c0 0x5643f6c6f7b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 >> 172.21.15.97:0/658698699 conn(0x5643f6bf9d10 msgr2=0x5643f6c5ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 shutdown_connections 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.862+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/658698699 wait complete. 2024-04-26T21:32:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.863+0000 7f9c3d5ee740 20 client.15099 trim_cache size 0 max 16384 2024-04-26T21:32:36.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.882+0000 7f9c3d5ee740 1 Processor -- start 2024-04-26T21:32:36.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.882+0000 7f9c3d5ee740 1 -- start start 2024-04-26T21:32:36.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.882+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c652c0 0x5643f6c6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.882+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7b700 con 0x5643f6c65cb0 2024-04-26T21:32:36.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7bdd0 con 0x5643f6c652c0 2024-04-26T21:32:36.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7c4a0 con 0x5643f6c6ae20 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38188/0 (socket says 172.21.15.97:38188) 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 -- 172.21.15.97:0/1904595336 learned_addr learned my addr 172.21.15.97:0/1904595336 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c33e00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c652c0 0x5643f6c6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c33e00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 -- 172.21.15.97:0/1904595336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 msgr2=0x5643f6c84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 -- 172.21.15.97:0/1904595336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c652c0 msgr2=0x5643f6c6fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c652c0 0x5643f6c6fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c32a00640 1 -- 172.21.15.97:0/1904595336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f6c5c460 con 0x5643f6c65cb0 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.883+0000 7f9c33e00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.884+0000 7f9c32a00640 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5643f6c70c20 tx=0x7f9c2000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2728257cdcca36f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.884+0000 7f9c1fe00640 1 -- 172.21.15.97:0/1904595336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20011950 con 0x5643f6c65cb0 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.884+0000 7f9c1fe00640 1 -- 172.21.15.97:0/1904595336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c20018310 con 0x5643f6c65cb0 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.884+0000 7f9c1fe00640 1 -- 172.21.15.97:0/1904595336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200106c0 con 0x5643f6c65cb0 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/1904595336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 msgr2=0x5643f6c76c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5643f6c70c20 tx=0x7f9c2000df60 comp rx=0 tx=0).stop 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/1904595336 shutdown_connections 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c6ae20 0x5643f6c84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c65cb0 0x5643f6c76c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/1904595336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c652c0 0x5643f6c6fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/1904595336 >> 172.21.15.97:0/1904595336 conn(0x5643f6bf9d10 msgr2=0x5643f6c6f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/1904595336 shutdown_connections 2024-04-26T21:32:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/1904595336 wait complete. 2024-04-26T21:32:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 -- start start 2024-04-26T21:32:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.885+0000 7f9c3d5ee740 1 Processor -- start 2024-04-26T21:32:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c652c0 0x5643f6c72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7d240 con 0x5643f6c6ae20 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7cb70 con 0x5643f6c65cb0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643f6c7d910 con 0x5643f6c652c0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c3d5ee740 10 client.0 _ll_register_callbacks cb 0x7f9c2c009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38200/0 (socket says 172.21.15.97:38200) 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c33e00640 1 -- 172.21.15.97:0/2572210870 learned_addr learned my addr 172.21.15.97:0/2572210870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c32a00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.886+0000 7f9c33e00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c652c0 0x5643f6c72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 -- 172.21.15.97:0/2572210870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c652c0 msgr2=0x5643f6c72eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c652c0 0x5643f6c72eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 -- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 msgr2=0x5643f6c733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 -- 172.21.15.97:0/2572210870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f6c76370 con 0x5643f6c6ae20 2024-04-26T21:32:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c33e00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01a550 tx=0x7f9c2c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dded07f871be06c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c033040 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9c2c01a5d0 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c20018310 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.887+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c01a7d0 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643f6ce22a0 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c20018310 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c2c0127f0 con 0x5643f6c6ae20 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 10 client.15105 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 20 client.15105 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6ce6e60 0x5643f6ce32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 4 client.15105 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.888+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 --> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] -- command(tid 0: {"prefix": "status"}) -- 0x5643f6bd1080 con 0x5643f6ce6e60 2024-04-26T21:32:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.889+0000 7f9c32a00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6ce6e60 0x5643f6ce32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:36.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c1d600640 10 client.15105 ms_handle_connect on v2:172.21.15.111:6832/4057011733 2024-04-26T21:32:36.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c32a00640 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6ce6e60 0x5643f6ce32a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e484e5be8ffea11b server_cookie=3ae011a8adcf558b in_seq=0 out_seq=0 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c1d600640 1 -- 172.21.15.97:0/2572210870 <== mds.3 v2:172.21.15.111:6832/4057011733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5643f6bd1080 con 0x5643f6ce6e60 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c1d600640 10 client.15105 handle_command_reply: tid=0 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 client.15105 shutdown 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 msgr2=0x5643f6c75db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01a550 tx=0x7f9c2c009a50 comp rx=0 tx=0).stop 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 shutdown_connections 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x5643f6ce6e60 0x5643f6ce32a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643f6c6ae20 0x5643f6c75db0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c1d600640 0 client.15105 ms_handle_reset on v2:172.21.15.111:6832/4057011733 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643f6c65cb0 0x5643f6c733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 --2- 172.21.15.97:0/2572210870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643f6c652c0 0x5643f6c72eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.890+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 >> 172.21.15.97:0/2572210870 conn(0x5643f6bf9d10 msgr2=0x5643f6c63c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 shutdown_connections 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c3d5ee740 1 -- 172.21.15.97:0/2572210870 wait complete. 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c3d5ee740 20 client.15105 trim_cache size 0 max 16384 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34400, 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 97.582650780999998, 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:32:36.895 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 149.873545068 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 msgr2=0x7f9c180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 0x7f9c180a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5643f6c66720 tx=0x7f9c24009260 comp rx=0 tx=0).stop 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 msgr2=0x7f9c3410c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9c24031460 tx=0x7f9c24003c80 comp rx=0 tx=0).stop 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 shutdown_connections 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c34110c70 0x7f9c3410b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3410a500 0x7f9c3410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9c180a3c40 0x7f9c180a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 --2- 172.21.15.97:0/3380726227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c34073e20 0x7f9c3410c5c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.891+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 >> 172.21.15.97:0/3380726227 conn(0x7f9c3406f750 msgr2=0x7f9c3410cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.892+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 shutdown_connections 2024-04-26T21:32:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:36.892+0000 7f9c39200640 1 -- 172.21.15.97:0/3380726227 wait complete. 2024-04-26T21:32:36.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/664482677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:36.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3321764038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:36.910 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:32:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 -- 172.21.15.97:0/1644376608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690104990 msgr2=0x7f2690104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 --2- 172.21.15.97:0/1644376608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690104990 0x7f2690104db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2680009900 tx=0x7f268002a410 comp rx=0 tx=0).stop 2024-04-26T21:32:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 -- 172.21.15.97:0/1644376608 shutdown_connections 2024-04-26T21:32:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 --2- 172.21.15.97:0/1644376608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2690106540 0x7f269010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 --2- 172.21.15.97:0/1644376608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690105b80 0x7f2690106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 --2- 172.21.15.97:0/1644376608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690104990 0x7f2690104db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 -- 172.21.15.97:0/1644376608 >> 172.21.15.97:0/1644376608 conn(0x7f26900fff00 msgr2=0x7f2690102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 -- 172.21.15.97:0/1644376608 shutdown_connections 2024-04-26T21:32:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.107+0000 7f2695800640 1 -- 172.21.15.97:0/1644376608 wait complete. 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.108+0000 7f2695800640 1 Processor -- start 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.108+0000 7f2695800640 1 -- start start 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.108+0000 7f2695800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690104990 0x7f269019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.108+0000 7f2695800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.108+0000 7f2695800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2690106540 0x7f26901a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2695800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26901103c0 con 0x7f2690105b80 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2695800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110240 con 0x7f2690104990 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2695800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110540 con 0x7f2690106540 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38220/0 (socket says 172.21.15.97:38220) 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 -- 172.21.15.97:0/877415869 learned_addr learned my addr 172.21.15.97:0/877415869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f268fe00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690104990 0x7f269019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 -- 172.21.15.97:0/877415869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2690106540 msgr2=0x7f26901a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2690106540 0x7f26901a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 -- 172.21.15.97:0/877415869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690104990 msgr2=0x7f269019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690104990 0x7f269019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 -- 172.21.15.97:0/877415869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901a2700 con 0x7f2690105b80 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.109+0000 7f2687e00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f267800d950 tx=0x7f267800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.110+0000 7f268d600640 1 -- 172.21.15.97:0/877415869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678017070 con 0x7f2690105b80 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.110+0000 7f268d600640 1 -- 172.21.15.97:0/877415869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2678009c30 con 0x7f2690105b80 2024-04-26T21:32:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.110+0000 7f268d600640 1 -- 172.21.15.97:0/877415869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267800f620 con 0x7f2690105b80 2024-04-26T21:32:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.110+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901a29f0 con 0x7f2690105b80 2024-04-26T21:32:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.110+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a2ef0 con 0x7f2690105b80 2024-04-26T21:32:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 Processor -- start 2024-04-26T21:32:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 -- start start 2024-04-26T21:32:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f268d600640 1 -- 172.21.15.97:0/877415869 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2678010460 con 0x7f2690105b80 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a0402c0 0x55f15a0406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f159fa4600 con 0x55f15a040cb0 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f159f41100 con 0x55f15a045e20 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.113+0000 7f2699c84740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f159fa05c0 con 0x55f15a0402c0 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38224/0 (socket says 172.21.15.97:38224) 2024-04-26T21:32:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 -- 172.21.15.97:0/4287682163 learned_addr learned my addr 172.21.15.97:0/4287682163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f2694e00640 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a0402c0 0x55f15a0406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f2687e00640 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 -- 172.21.15.97:0/4287682163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a0402c0 msgr2=0x55f15a0406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a0402c0 0x55f15a0406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 -- 172.21.15.97:0/4287682163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 msgr2=0x55f15a0482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 -- 172.21.15.97:0/4287682163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f159cb62d0 con 0x55f15a040cb0 2024-04-26T21:32:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.114+0000 7f268fe00640 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26800040f0 tx=0x7f2680004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc016daf13a6313 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2685600640 1 -- 172.21.15.97:0/4287682163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680040070 con 0x55f15a040cb0 2024-04-26T21:32:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2685600640 1 -- 172.21.15.97:0/4287682163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680003a80 con 0x55f15a040cb0 2024-04-26T21:32:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2685600640 1 -- 172.21.15.97:0/4287682163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680035030 con 0x55f15a040cb0 2024-04-26T21:32:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f268d600640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 0x7f26740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f268d600640 1 -- 172.21.15.97:0/877415869 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f26780d01c0 con 0x7f2690105b80 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2694e00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 0x7f26740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2699c84740 1 -- 172.21.15.97:0/4287682163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 msgr2=0x55f15a0457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.115+0000 7f2699c84740 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26800040f0 tx=0x7f2680004120 comp rx=0 tx=0).stop 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 -- 172.21.15.97:0/4287682163 shutdown_connections 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a0457b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 --2- 172.21.15.97:0/4287682163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a0402c0 0x55f15a0406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 -- 172.21.15.97:0/4287682163 >> 172.21.15.97:0/4287682163 conn(0x55f159fd4d10 msgr2=0x55f15a03bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 -- 172.21.15.97:0/4287682163 shutdown_connections 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 -- 172.21.15.97:0/4287682163 wait complete. 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2694e00640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 0x7f26740a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2688009270 tx=0x7f2688009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.116+0000 7f2699c84740 1 -- start start 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 Processor -- start 2024-04-26T21:32:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a040cb0 0x55f15a04ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a051c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f15a048d70 con 0x55f15a0402c0 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f159f2bf70 con 0x55f15a040cb0 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f15a048ef0 con 0x55f15a045e20 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f2699c84740 10 client.0 _ll_register_callbacks cb 0x55f159f9a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38228/0 (socket says 172.21.15.97:38228) 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.117+0000 7f268fe00640 1 -- 172.21.15.97:0/4100474313 learned_addr learned my addr 172.21.15.97:0/4100474313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f2687e00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a040cb0 0x55f15a04ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 -- 172.21.15.97:0/4100474313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 msgr2=0x55f15a051c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a051c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 -- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a040cb0 msgr2=0x55f15a04ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a040cb0 0x55f15a04ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 -- 172.21.15.97:0/4100474313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f159d806b0 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f268fe00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26800092a0 tx=0x7f2680003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3efd98f1fd550c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f267e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268004d070 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f26800319e0 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2680003a80 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2680040070 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.118+0000 7f2699c84740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f15a054c40 con 0x55f15a0402c0 2024-04-26T21:32:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680003a80 con 0x55f15a0402c0 2024-04-26T21:32:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2680050030 con 0x55f15a0402c0 2024-04-26T21:32:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 10 client.15117 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:32:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 20 client.15117 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a056e60 0x55f15a0582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 4 client.15117 mds_command: new command op to 34433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.119+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 --> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f159fb2210 con 0x55f15a056e60 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.120+0000 7f2687e00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a056e60 0x55f15a0582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.121+0000 7f2687e00640 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a056e60 0x55f15a0582d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d0731dd929b70ca server_cookie=988aefa2b0cc1622 in_seq=0 out_seq=0 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.121+0000 7f267e000640 10 client.15117 ms_handle_connect on v2:172.21.15.111:6834/738887742 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f267e000640 1 -- 172.21.15.97:0/4100474313 <== mds.2 v2:172.21.15.111:6834/738887742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f159fb2210 con 0x55f15a056e60 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f267e000640 10 client.15117 handle_command_reply: tid=0 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 client.15117 shutdown 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 msgr2=0x55f15a04a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26800092a0 tx=0x7f2680003e60 comp rx=0 tx=0).stop 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 shutdown_connections 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a056e60 0x55f15a0582d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a051c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a040cb0 0x55f15a04ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f267e000640 0 client.15117 ms_handle_reset on v2:172.21.15.111:6834/738887742 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 --2- 172.21.15.97:0/4100474313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a04a6e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 >> 172.21.15.97:0/4100474313 conn(0x55f159fd4d10 msgr2=0x55f15a03af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 shutdown_connections 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 1 -- 172.21.15.97:0/4100474313 wait complete. 2024-04-26T21:32:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.123+0000 7f2699c84740 20 client.15117 trim_cache size 0 max 16384 2024-04-26T21:32:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:36 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/664482677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:36 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3321764038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.143+0000 7f2699c84740 1 Processor -- start 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.143+0000 7f2699c84740 1 -- start start 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.143+0000 7f2699c84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.143+0000 7f2699c84740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a040cb0 0x55f15a051c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f2699c84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f2699c84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f15a056610 con 0x55f15a0402c0 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f2699c84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f15a056ce0 con 0x55f15a045e20 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f2699c84740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f15a0573b0 con 0x55f15a040cb0 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38238/0 (socket says 172.21.15.97:38238) 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 -- 172.21.15.97:0/2919629992 learned_addr learned my addr 172.21.15.97:0/2919629992 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 -- 172.21.15.97:0/2919629992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a040cb0 msgr2=0x55f15a051c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a040cb0 0x55f15a051c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 -- 172.21.15.97:0/2919629992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 msgr2=0x55f15a0492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 -- 172.21.15.97:0/2919629992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15a0453b0 con 0x55f15a0402c0 2024-04-26T21:32:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.144+0000 7f268fe00640 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f268002a930 tx=0x7f2680004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6822ba10df41ed54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2684c00640 1 -- 172.21.15.97:0/2919629992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268004d070 con 0x55f15a0402c0 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2684c00640 1 -- 172.21.15.97:0/2919629992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680031600 con 0x55f15a0402c0 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2684c00640 1 -- 172.21.15.97:0/2919629992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680035040 con 0x55f15a0402c0 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 -- 172.21.15.97:0/2919629992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 msgr2=0x55f15a048210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f268002a930 tx=0x7f2680004500 comp rx=0 tx=0).stop 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 -- 172.21.15.97:0/2919629992 shutdown_connections 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a045e20 0x55f15a0492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a040cb0 0x55f15a051c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 --2- 172.21.15.97:0/2919629992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a0402c0 0x55f15a048210 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.145+0000 7f2699c84740 1 -- 172.21.15.97:0/2919629992 >> 172.21.15.97:0/2919629992 conn(0x55f159fd4d10 msgr2=0x55f15a03a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 -- 172.21.15.97:0/2919629992 shutdown_connections 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 -- 172.21.15.97:0/2919629992 wait complete. 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 -- start start 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 Processor -- start 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a0402c0 0x55f15a04d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.146+0000 7f2699c84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2699c84740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a0509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2699c84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f15a058150 con 0x55f15a040cb0 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2699c84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f15a057a80 con 0x55f15a0402c0 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2699c84740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f15a058820 con 0x55f15a045e20 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2699c84740 10 client.0 _ll_register_callbacks cb 0x7f2680031500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38252/0 (socket says 172.21.15.97:38252) 2024-04-26T21:32:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 -- 172.21.15.97:0/1087067030 learned_addr learned my addr 172.21.15.97:0/1087067030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f268fe00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a0402c0 0x55f15a04d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 -- 172.21.15.97:0/1087067030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 msgr2=0x55f15a0509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a0509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 -- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a0402c0 msgr2=0x55f15a04d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a0402c0 0x55f15a04d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 -- 172.21.15.97:0/1087067030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15a050fa0 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f2687e00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2678098d20 tx=0x7f267809c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3facaf3ffbb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f267e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.147+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678020040 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f267809b7a0 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2680031600 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26780a0700 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f15a0c57b0 con 0x55f15a040cb0 2024-04-26T21:32:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680031600 con 0x55f15a040cb0 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2678098230 con 0x55f15a040cb0 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 10 client.15123 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 20 client.15123 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a0ca3d0 0x55f15a0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 4 client.15123 mds_command: new command op to 34433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:32:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.148+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 --> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] -- command(tid 0: {"prefix": "status"}) -- 0x55f159fac080 con 0x55f15a0ca3d0 2024-04-26T21:32:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.149+0000 7f268fe00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a0ca3d0 0x55f15a0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f268fe00640 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a0ca3d0 0x55f15a0c90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccb8f7e854570531 server_cookie=9a119a7b8b2643c7 in_seq=0 out_seq=0 2024-04-26T21:32:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f267e000640 10 client.15123 ms_handle_connect on v2:172.21.15.111:6834/738887742 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f267e000640 1 -- 172.21.15.97:0/1087067030 <== mds.2 v2:172.21.15.111:6834/738887742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f159fac080 con 0x55f15a0ca3d0 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f267e000640 10 client.15123 handle_command_reply: tid=0 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f2699c84740 1 client.15123 shutdown 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.150+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 msgr2=0x55f15a04d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2678098d20 tx=0x7f267809c240 comp rx=0 tx=0).stop 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 shutdown_connections 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x55f15a0ca3d0 0x55f15a0c90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f15a045e20 0x55f15a0509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f15a040cb0 0x55f15a04d9e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 --2- 172.21.15.97:0/1087067030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f15a0402c0 0x55f15a04d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f267e000640 0 client.15123 ms_handle_reset on v2:172.21.15.111:6834/738887742 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 >> 172.21.15.97:0/1087067030 conn(0x55f159fd4d10 msgr2=0x55f15a03a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 shutdown_connections 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 1 -- 172.21.15.97:0/1087067030 wait complete. 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2699c84740 20 client.15123 trim_cache size 0 max 16384 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34433, 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 98.860048172999996, 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:32:37.155 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 138.18485706199999 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 msgr2=0x7f26740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 0x7f26740a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2688009270 tx=0x7f2688009500 comp rx=0 tx=0).stop 2024-04-26T21:32:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 msgr2=0x7f269019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.151+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f267800d950 tx=0x7f267800de10 comp rx=0 tx=0).stop 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 shutdown_connections 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2690106540 0x7f26901a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26740a3cc0 0x7f26740a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690105b80 0x7f269019b020 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 --2- 172.21.15.97:0/877415869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2690104990 0x7f269019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 >> 172.21.15.97:0/877415869 conn(0x7f26900fff00 msgr2=0x7f2690108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 shutdown_connections 2024-04-26T21:32:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.152+0000 7f2695800640 1 -- 172.21.15.97:0/877415869 wait complete. 2024-04-26T21:32:37.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:32:37.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:36 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/664482677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:32:37.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:36 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3321764038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:32:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 -- 172.21.15.97:0/1775699065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd8109f80 msgr2=0x7f8bd810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 --2- 172.21.15.97:0/1775699065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd8109f80 0x7f8bd810a400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009900 tx=0x7f8bc402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 -- 172.21.15.97:0/1775699065 shutdown_connections 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 --2- 172.21.15.97:0/1775699065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd810a940 0x7f8bd81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 --2- 172.21.15.97:0/1775699065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd8109f80 0x7f8bd810a400 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 --2- 172.21.15.97:0/1775699065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8108d90 0x7f8bd81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.347+0000 7f8bdd400640 1 -- 172.21.15.97:0/1775699065 >> 172.21.15.97:0/1775699065 conn(0x7f8bd8077d30 msgr2=0x7f8bd807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.348+0000 7f8bdd400640 1 -- 172.21.15.97:0/1775699065 shutdown_connections 2024-04-26T21:32:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.348+0000 7f8bdd400640 1 -- 172.21.15.97:0/1775699065 wait complete. 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.348+0000 7f8bdd400640 1 Processor -- start 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 -- start start 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 0x7f8bd819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8109f80 0x7f8bd819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8114820 con 0x7f8bd810a940 2024-04-26T21:32:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81146a0 con 0x7f8bd8109f80 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdd400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81149a0 con 0x7f8bd8108d90 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38268/0 (socket says 172.21.15.97:38268) 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bdca00640 1 -- 172.21.15.97:0/3381623237 learned_addr learned my addr 172.21.15.97:0/3381623237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.349+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 0x7f8bd819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bd7400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8109f80 0x7f8bd819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 -- 172.21.15.97:0/3381623237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 msgr2=0x7f8bd819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 0x7f8bd819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 -- 172.21.15.97:0/3381623237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8109f80 msgr2=0x7f8bd819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8109f80 0x7f8bd819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 -- 172.21.15.97:0/3381623237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd81a6ae0 con 0x7f8bd810a940 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 0x7f8bd819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdca00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b970 tx=0x7f8bd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bd4c00640 1 -- 172.21.15.97:0/3381623237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0019070 con 0x7f8bd810a940 2024-04-26T21:32:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bd4c00640 1 -- 172.21.15.97:0/3381623237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bd0014440 con 0x7f8bd810a940 2024-04-26T21:32:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bd4c00640 1 -- 172.21.15.97:0/3381623237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00075d0 con 0x7f8bd810a940 2024-04-26T21:32:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd81a6dd0 con 0x7f8bd810a940 2024-04-26T21:32:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.350+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a72c0 con 0x7f8bd810a940 2024-04-26T21:32:37.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.353+0000 7f8be1842740 1 Processor -- start 2024-04-26T21:32:37.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.353+0000 7f8bd4c00640 1 -- 172.21.15.97:0/3381623237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8bd001c070 con 0x7f8bd810a940 2024-04-26T21:32:37.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 -- start start 2024-04-26T21:32:37.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb124600 con 0x55cbcb1c5e20 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb0c1100 con 0x55cbcb1c0cb0 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8be1842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1205c0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8bd7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8bd7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38276/0 (socket says 172.21.15.97:38276) 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8bd7400640 1 -- 172.21.15.97:0/4055273509 learned_addr learned my addr 172.21.15.97:0/4055273509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.354+0000 7f8bd7400640 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 -- 172.21.15.97:0/4055273509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c02c0 msgr2=0x55cbcb1c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 -- 172.21.15.97:0/4055273509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 msgr2=0x55cbcb1c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 -- 172.21.15.97:0/4055273509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcae362d0 con 0x55cbcb1c5e20 2024-04-26T21:32:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7400640 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4005ec0 tx=0x7f8bc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321a4c58f0ac536b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd4c00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 0x7f8bbc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bcd200640 1 -- 172.21.15.97:0/4055273509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4043020 con 0x55cbcb1c5e20 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bcd200640 1 -- 172.21.15.97:0/4055273509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bc4003d00 con 0x55cbcb1c5e20 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bcd200640 1 -- 172.21.15.97:0/4055273509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4035070 con 0x55cbcb1c5e20 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 0x7f8bbc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.355+0000 7f8bd4c00640 1 -- 172.21.15.97:0/3381623237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8bd00cf8a0 con 0x7f8bd810a940 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 -- 172.21.15.97:0/4055273509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 msgr2=0x55cbcb1c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 0x7f8bbc0a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8005f20 tx=0x7f8bc8005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4005ec0 tx=0x7f8bc4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 -- 172.21.15.97:0/4055273509 shutdown_connections 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c82b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 --2- 172.21.15.97:0/4055273509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 -- 172.21.15.97:0/4055273509 >> 172.21.15.97:0/4055273509 conn(0x55cbcb154d10 msgr2=0x55cbcb1bbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 -- 172.21.15.97:0/4055273509 shutdown_connections 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.356+0000 7f8be1842740 1 -- 172.21.15.97:0/4055273509 wait complete. 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.357+0000 7f8be1842740 1 -- start start 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.357+0000 7f8be1842740 1 Processor -- start 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.357+0000 7f8be1842740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.357+0000 7f8be1842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.357+0000 7f8be1842740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8be1842740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1c8d70 con 0x55cbcb1c02c0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8be1842740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb0abf70 con 0x55cbcb1c5e20 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8be1842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1c8ef0 con 0x55cbcb1c0cb0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8be1842740 10 client.0 _ll_register_callbacks cb 0x55cbcb11a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38286/0 (socket says 172.21.15.97:38286) 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 -- 172.21.15.97:0/49066030 learned_addr learned my addr 172.21.15.97:0/49066030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bdca00640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7400640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 -- 172.21.15.97:0/49066030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 msgr2=0x55cbcb1cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 -- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 msgr2=0x55cbcb1d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bd7e00640 1 -- 172.21.15.97:0/49066030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcaf006b0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.358+0000 7f8bdca00640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8109de0 tx=0x7f8bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb4fd447deeb83b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bc1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc801d420 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8bc801d5c0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc4003d00 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bc80047e0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8be1842740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbcb1d4d40 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.359+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bc4003d00 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bc80054b0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 20 client.15135 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb1d74a0 0x55cbcb1d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 4 client.15135 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:32:37.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.360+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 --> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbcb132210 con 0x55cbcb1d74a0 2024-04-26T21:32:37.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.361+0000 7f8bd7400640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb1d74a0 0x55cbcb1d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.362+0000 7f8bd7400640 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb1d74a0 0x55cbcb1d8910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dc8ceca5ddfd36d server_cookie=90ebf8a2a47366ab in_seq=0 out_seq=0 2024-04-26T21:32:37.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.362+0000 7f8bc1600640 10 client.15135 ms_handle_connect on v2:172.21.15.111:6836/1944730061 2024-04-26T21:32:37.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.364+0000 7f8bc1600640 1 -- 172.21.15.97:0/49066030 <== mds.1 v2:172.21.15.111:6836/1944730061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbcb132210 con 0x55cbcb1d74a0 2024-04-26T21:32:37.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.364+0000 7f8bc1600640 10 client.15135 handle_command_reply: tid=0 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.364+0000 7f8be1842740 1 client.15135 shutdown 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 msgr2=0x55cbcb1ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8109de0 tx=0x7f8bc80046c0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 shutdown_connections 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb1d74a0 0x55cbcb1d8910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 --2- 172.21.15.97:0/49066030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1ca710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 >> 172.21.15.97:0/49066030 conn(0x55cbcb154d10 msgr2=0x55cbcb1baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8bc1600640 0 client.15135 ms_handle_reset on v2:172.21.15.111:6836/1944730061 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 shutdown_connections 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 1 -- 172.21.15.97:0/49066030 wait complete. 2024-04-26T21:32:37.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.365+0000 7f8be1842740 20 client.15135 trim_cache size 0 max 16384 2024-04-26T21:32:37.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.384+0000 7f8be1842740 1 Processor -- start 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 -- start start 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d6c50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d7320 con 0x55cbcb1c5e20 2024-04-26T21:32:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8be1842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d79f0 con 0x55cbcb1c0cb0 2024-04-26T21:32:37.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38296/0 (socket says 172.21.15.97:38296) 2024-04-26T21:32:37.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.385+0000 7f8bd7e00640 1 -- 172.21.15.97:0/3401854578 learned_addr learned my addr 172.21.15.97:0/3401854578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bdca00640 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7400640 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 -- 172.21.15.97:0/3401854578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 msgr2=0x55cbcb1d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 -- 172.21.15.97:0/3401854578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 msgr2=0x55cbcb1c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 -- 172.21.15.97:0/3401854578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcb1c1090 con 0x55cbcb1c02c0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e000 tx=0x7f8bc8010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c792fcf1e7f05128 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bc3e00640 1 -- 172.21.15.97:0/3401854578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800bda0 con 0x55cbcb1c02c0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bc3e00640 1 -- 172.21.15.97:0/3401854578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bc801de50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.386+0000 7f8bc3e00640 1 -- 172.21.15.97:0/3401854578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8005000 con 0x55cbcb1c02c0 2024-04-26T21:32:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 -- 172.21.15.97:0/3401854578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 msgr2=0x55cbcb1d6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e000 tx=0x7f8bc8010b30 comp rx=0 tx=0).stop 2024-04-26T21:32:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 -- 172.21.15.97:0/3401854578 shutdown_connections 2024-04-26T21:32:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 --2- 172.21.15.97:0/3401854578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1d6200 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 -- 172.21.15.97:0/3401854578 >> 172.21.15.97:0/3401854578 conn(0x55cbcb154d10 msgr2=0x55cbcb1c1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 -- 172.21.15.97:0/3401854578 shutdown_connections 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.387+0000 7f8be1842740 1 -- 172.21.15.97:0/3401854578 wait complete. 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 -- start start 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 Processor -- start 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d8790 con 0x55cbcb1c02c0 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d80c0 con 0x55cbcb1c0cb0 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbcb1d8e60 con 0x55cbcb1c5e20 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8be1842740 10 client.0 _ll_register_callbacks cb 0x7f8bc800b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.388+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38298/0 (socket says 172.21.15.97:38298) 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 -- 172.21.15.97:0/340763248 learned_addr learned my addr 172.21.15.97:0/340763248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 -- 172.21.15.97:0/340763248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c5e20 msgr2=0x55cbcb1d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bdca00640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7400640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 -- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 msgr2=0x55cbcb1cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 -- 172.21.15.97:0/340763248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcb1d0c50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bd7e00640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bb60 tx=0x7f8bc801dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bcce4b1226c83c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bc1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8040070 con 0x55cbcb1c02c0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.389+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8bc8004b90 con 0x55cbcb1c02c0 2024-04-26T21:32:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc801de50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bc804c050 con 0x55cbcb1c02c0 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbcb1d1f50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bc801de50 con 0x55cbcb1c02c0 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bc8030610 con 0x55cbcb1c02c0 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 20 client.15141 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb241ac0 0x55cbcb2407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 4 client.15141 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:32:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.390+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 --> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] -- command(tid 0: {"prefix": "status"}) -- 0x55cbcb12c080 con 0x55cbcb241ac0 2024-04-26T21:32:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.391+0000 7f8bd7400640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb241ac0 0x55cbcb2407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.392+0000 7f8bd7400640 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb241ac0 0x55cbcb2407d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6a4b9dd16bc996b server_cookie=cf403c3cc76805b6 in_seq=0 out_seq=0 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.392+0000 7f8bc1600640 10 client.15141 ms_handle_connect on v2:172.21.15.111:6836/1944730061 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.392+0000 7f8bc1600640 1 -- 172.21.15.97:0/340763248 <== mds.1 v2:172.21.15.111:6836/1944730061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cbcb12c080 con 0x55cbcb241ac0 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.392+0000 7f8bc1600640 10 client.15141 handle_command_reply: tid=0 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 client.15141 shutdown 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 msgr2=0x55cbcb1cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bb60 tx=0x7f8bc801dbf0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 shutdown_connections 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbcb241ac0 0x55cbcb2407d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbcb1c5e20 0x55cbcb1d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbcb1c0cb0 0x55cbcb1cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 --2- 172.21.15.97:0/340763248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbcb1c02c0 0x55cbcb1cd170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8bc1600640 0 client.15141 ms_handle_reset on v2:172.21.15.111:6836/1944730061 2024-04-26T21:32:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 >> 172.21.15.97:0/340763248 conn(0x55cbcb154d10 msgr2=0x55cbcb1d6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 shutdown_connections 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 1 -- 172.21.15.97:0/340763248 wait complete. 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.393+0000 7f8be1842740 20 client.15141 trim_cache size 0 max 16384 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34460, 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 100.11849429199999, 2024-04-26T21:32:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 126.19363036599999 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 msgr2=0x7f8bbc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 0x7f8bbc0a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8005f20 tx=0x7f8bc8005eb0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 msgr2=0x7f8bd81a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b970 tx=0x7f8bd000be30 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 shutdown_connections 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8bbc0a3cc0 0x7f8bbc0a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bd810a940 0x7f8bd81a6470 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd8109f80 0x7f8bd819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 --2- 172.21.15.97:0/3381623237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bd8108d90 0x7f8bd819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 >> 172.21.15.97:0/3381623237 conn(0x7f8bd8077d30 msgr2=0x7f8bd810d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 shutdown_connections 2024-04-26T21:32:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.394+0000 7f8bdd400640 1 -- 172.21.15.97:0/3381623237 wait complete. 2024-04-26T21:32:37.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 -- 172.21.15.97:0/331767123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2664108d90 msgr2=0x7f26641091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 --2- 172.21.15.97:0/331767123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2664108d90 0x7f26641091b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2658009ca0 tx=0x7f265802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 -- 172.21.15.97:0/331767123 shutdown_connections 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 --2- 172.21.15.97:0/331767123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f266410a940 0x7f26641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 --2- 172.21.15.97:0/331767123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 0x7f266410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 --2- 172.21.15.97:0/331767123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2664108d90 0x7f26641091b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 -- 172.21.15.97:0/331767123 >> 172.21.15.97:0/331767123 conn(0x7f2664077d30 msgr2=0x7f266407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 -- 172.21.15.97:0/331767123 shutdown_connections 2024-04-26T21:32:37.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.607+0000 7f266a200640 1 -- 172.21.15.97:0/331767123 wait complete. 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.608+0000 7f266a200640 1 Processor -- start 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.608+0000 7f266a200640 1 -- start start 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.608+0000 7f266a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2664108d90 0x7f266419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.608+0000 7f266a200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 0x7f266419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.608+0000 7f266a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f266a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26641147c0 con 0x7f266410a940 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f266a200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2664114640 con 0x7f2664109f80 2024-04-26T21:32:37.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f266a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2664114940 con 0x7f2664108d90 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38304/0 (socket says 172.21.15.97:38304) 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 -- 172.21.15.97:0/718338194 learned_addr learned my addr 172.21.15.97:0/718338194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2668e00640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2664108d90 0x7f266419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2663e00640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 0x7f266419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 -- 172.21.15.97:0/718338194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2664108d90 msgr2=0x7f266419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2664108d90 0x7f266419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 -- 172.21.15.97:0/718338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 msgr2=0x7f266419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 0x7f266419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.609+0000 7f2669800640 1 -- 172.21.15.97:0/718338194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641a6ae0 con 0x7f266410a940 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f2669800640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f265c00d750 tx=0x7f265c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f2661600640 1 -- 172.21.15.97:0/718338194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c017070 con 0x7f266410a940 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f2661600640 1 -- 172.21.15.97:0/718338194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c0054b0 con 0x7f266410a940 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641a6dd0 con 0x7f266410a940 2024-04-26T21:32:37.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f2661600640 1 -- 172.21.15.97:0/718338194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c004b90 con 0x7f266410a940 2024-04-26T21:32:37.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.610+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a7290 con 0x7f266410a940 2024-04-26T21:32:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.613+0000 7f266e779740 1 Processor -- start 2024-04-26T21:32:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.613+0000 7f2661600640 1 -- 172.21.15.97:0/718338194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f265c004d30 con 0x7f266410a940 2024-04-26T21:32:37.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 -- start start 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc058600 con 0x55a3bc0f42c0 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bbff5100 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f266e779740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc0545c0 con 0x55a3bc0f9e20 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f2663e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f2663e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38310/0 (socket says 172.21.15.97:38310) 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.614+0000 7f2663e00640 1 -- 172.21.15.97:0/3762781319 learned_addr learned my addr 172.21.15.97:0/3762781319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2668e00640 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 -- 172.21.15.97:0/3762781319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 msgr2=0x55a3bc0fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 -- 172.21.15.97:0/3762781319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 msgr2=0x55a3bc0f97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0f97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 -- 172.21.15.97:0/3762781319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bbd6a2d0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2663e00640 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f265000bd30 tx=0x7f2650011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb959caa4b1150 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2661600640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 0x7f264c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2656000640 1 -- 172.21.15.97:0/3762781319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265001a400 con 0x55a3bc0f42c0 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2656000640 1 -- 172.21.15.97:0/3762781319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265001aa30 con 0x55a3bc0f42c0 2024-04-26T21:32:37.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.615+0000 7f2656000640 1 -- 172.21.15.97:0/3762781319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650012930 con 0x55a3bc0f42c0 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f2661600640 1 -- 172.21.15.97:0/718338194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f265c0d0080 con 0x7f266410a940 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f2668e00640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 0x7f264c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 -- 172.21.15.97:0/3762781319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 msgr2=0x55a3bc0f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f265000bd30 tx=0x7f2650011750 comp rx=0 tx=0).stop 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 -- 172.21.15.97:0/3762781319 shutdown_connections 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 --2- 172.21.15.97:0/3762781319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f46e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 -- 172.21.15.97:0/3762781319 >> 172.21.15.97:0/3762781319 conn(0x55a3bc088d10 msgr2=0x55a3bc0efd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 -- 172.21.15.97:0/3762781319 shutdown_connections 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f2668e00640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 0x7f264c0a6140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2658000f80 tx=0x7f2658005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.616+0000 7f266e779740 1 -- 172.21.15.97:0/3762781319 wait complete. 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 -- start start 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 Processor -- start 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc105c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.617+0000 7f266e779740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc0fcdd0 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f266e779740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bbfdff70 con 0x55a3bc0f9e20 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f266e779740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc0fcf50 con 0x55a3bc0f42c0 2024-04-26T21:32:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f266e779740 10 client.0 _ll_register_callbacks cb 0x55a3bc04e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38322/0 (socket says 172.21.15.97:38322) 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 -- 172.21.15.97:0/3394717581 learned_addr learned my addr 172.21.15.97:0/3394717581 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2668e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 -- 172.21.15.97:0/3394717581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 msgr2=0x55a3bc0fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 -- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 msgr2=0x55a3bc105c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc105c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 -- 172.21.15.97:0/3394717581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bbe346b0 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2668e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.618+0000 7f2663e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2650004630 tx=0x7f265000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ac329743aef80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f264aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265001a810 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f2650005050 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265001aa30 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f265001ae20 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f266e779740 10 client.15153 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a3bc108c10 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.619+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265001aa30 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2650012670 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 20 client.15153 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc10afe0 0x55a3bc10c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 4 client.15153 mds_command: new command op to 34502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 --> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3bc066210 con 0x55a3bc10afe0 2024-04-26T21:32:37.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.620+0000 7f2668e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc10afe0 0x55a3bc10c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.621+0000 7f2668e00640 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc10afe0 0x55a3bc10c450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22b52f2abfad58b server_cookie=4300444a8107285c in_seq=0 out_seq=0 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.621+0000 7f264aa00640 10 client.15153 ms_handle_connect on v2:172.21.15.111:6838/2198422773 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f264aa00640 1 -- 172.21.15.97:0/3394717581 <== mds.0 v2:172.21.15.111:6838/2198422773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3bc066210 con 0x55a3bc10afe0 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f264aa00640 10 client.15153 handle_command_reply: tid=0 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 client.15153 shutdown 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 msgr2=0x55a3bc0febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2650004630 tx=0x7f265000b470 comp rx=0 tx=0).stop 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 shutdown_connections 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc10afe0 0x55a3bc10c450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc0febf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 --2- 172.21.15.97:0/3394717581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 >> 172.21.15.97:0/3394717581 conn(0x55a3bc088d10 msgr2=0x55a3bc0eef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f264aa00640 0 client.15153 ms_handle_reset on v2:172.21.15.111:6838/2198422773 2024-04-26T21:32:37.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 shutdown_connections 2024-04-26T21:32:37.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 1 -- 172.21.15.97:0/3394717581 wait complete. 2024-04-26T21:32:37.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.624+0000 7f266e779740 20 client.15153 trim_cache size 0 max 16384 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 Processor -- start 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 -- start start 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc105bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10a790 con 0x55a3bc0f42c0 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10ae60 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.644+0000 7f266e779740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10b530 con 0x55a3bc0f9e20 2024-04-26T21:32:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38334/0 (socket says 172.21.15.97:38334) 2024-04-26T21:32:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 -- 172.21.15.97:0/1138654940 learned_addr learned my addr 172.21.15.97:0/1138654940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2663e00640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc105bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2669800640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 -- 172.21.15.97:0/1138654940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 msgr2=0x55a3bc0f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 -- 172.21.15.97:0/1138654940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 msgr2=0x55a3bc105bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc105bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 -- 172.21.15.97:0/1138654940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bc0ea370 con 0x55a3bc0f42c0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2669800640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2668e00640 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2664109de0 tx=0x7f2658040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548be43006705b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.645+0000 7f2655600640 1 -- 172.21.15.97:0/1138654940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658053620 con 0x55a3bc0f42c0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f2655600640 1 -- 172.21.15.97:0/1138654940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2658036040 con 0x55a3bc0f42c0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f2655600640 1 -- 172.21.15.97:0/1138654940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580476b0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 -- 172.21.15.97:0/1138654940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 msgr2=0x55a3bc0f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2664109de0 tx=0x7f2658040000 comp rx=0 tx=0).stop 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 -- 172.21.15.97:0/1138654940 shutdown_connections 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f9e20 0x55a3bc0f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 --2- 172.21.15.97:0/1138654940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc0f96e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 -- 172.21.15.97:0/1138654940 >> 172.21.15.97:0/1138654940 conn(0x55a3bc088d10 msgr2=0x55a3bc0f54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 -- 172.21.15.97:0/1138654940 shutdown_connections 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.646+0000 7f266e779740 1 -- 172.21.15.97:0/1138654940 wait complete. 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 -- start start 2024-04-26T21:32:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 Processor -- start 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc16cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc16f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10c2d0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10bc00 con 0x55a3bc0f9e20 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a3bc10c9a0 con 0x55a3bc0f4cb0 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.647+0000 7f266e779740 10 client.0 _ll_register_callbacks cb 0x7f265000bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:32:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38336/0 (socket says 172.21.15.97:38336) 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 -- 172.21.15.97:0/1017019048 learned_addr learned my addr 172.21.15.97:0/1017019048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2663e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc16cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 -- 172.21.15.97:0/1017019048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f4cb0 msgr2=0x55a3bc16cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc16cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 -- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 msgr2=0x55a3bc16f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2669800640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc16f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc16f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 -- 172.21.15.97:0/1017019048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bc16fda0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f2668e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2658032ab0 tx=0x7f26580317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c5968fe52ae957 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.648+0000 7f264aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658053830 con 0x55a3bc0f42c0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f26580305c0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2658036040 con 0x55a3bc0f42c0 2024-04-26T21:32:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26580307c0 con 0x55a3bc0f42c0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f266e779740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a3bc171010 con 0x55a3bc0f42c0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2658036040 con 0x55a3bc0f42c0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2658050070 con 0x55a3bc0f42c0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.649+0000 7f266e779740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f266e779740 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f266e779740 20 client.15159 populate_metadata read hostname 'smithi097' 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc175940 0x55a3bc171d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f266e779740 4 client.15159 mds_command: new command op to 34502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 --> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] -- command(tid 0: {"prefix": "status"}) -- 0x55a3bc060080 con 0x55a3bc175940 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.650+0000 7f2663e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc175940 0x55a3bc171d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.651+0000 7f2663e00640 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc175940 0x55a3bc171d80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc15a19feff96cf6 server_cookie=2e6d22ecc46bd0c8 in_seq=0 out_seq=0 2024-04-26T21:32:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.651+0000 7f264aa00640 10 client.15159 ms_handle_connect on v2:172.21.15.111:6838/2198422773 2024-04-26T21:32:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f264aa00640 1 -- 172.21.15.97:0/1017019048 <== mds.0 v2:172.21.15.111:6838/2198422773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a3bc060080 con 0x55a3bc175940 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f264aa00640 10 client.15159 handle_command_reply: tid=0 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 client.15159 shutdown 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 msgr2=0x55a3bc103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2658032ab0 tx=0x7f26580317f0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 shutdown_connections 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x55a3bc175940 0x55a3bc171d80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bc0f9e20 0x55a3bc16f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a3bc0f4cb0 0x55a3bc16cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 --2- 172.21.15.97:0/1017019048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a3bc0f42c0 0x55a3bc103be0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f264aa00640 0 client.15159 ms_handle_reset on v2:172.21.15.111:6838/2198422773 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 >> 172.21.15.97:0/1017019048 conn(0x55a3bc088d10 msgr2=0x55a3bc0f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 shutdown_connections 2024-04-26T21:32:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 1 -- 172.21.15.97:0/1017019048 wait complete. 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.652+0000 7f266e779740 20 client.15159 trim_cache size 0 max 16384 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34502, 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 103.696674457, 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:32:37.657 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:32:37.658 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 110.521065851 2024-04-26T21:32:37.658 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:32:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 msgr2=0x7f264c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 0x7f264c0a6140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2658000f80 tx=0x7f2658005ef0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 msgr2=0x7f26641a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f265c00d750 tx=0x7f265c00dc10 comp rx=0 tx=0).stop 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 shutdown_connections 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f264c0a3c80 0x7f264c0a6140 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f266410a940 0x7f26641a6470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2664109f80 0x7f266419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 --2- 172.21.15.97:0/718338194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2664108d90 0x7f266419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 >> 172.21.15.97:0/718338194 conn(0x7f2664077d30 msgr2=0x7f266410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 shutdown_connections 2024-04-26T21:32:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:37.653+0000 7f266a200640 1 -- 172.21.15.97:0/718338194 wait complete. 2024-04-26T21:32:37.672 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T21:32:37.673 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:32:37.683 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:32:37.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T21:32:38.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:37 smithi097 ceph-mon[28127]: pgmap v418: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:38.139 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:37 smithi111 ceph-mon[31001]: pgmap v418: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:37 smithi143 ceph-mon[31461]: pgmap v418: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:32:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:39 smithi111 ceph-mon[31001]: pgmap v419: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:40.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:39 smithi097 ceph-mon[28127]: pgmap v419: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:39 smithi143 ceph-mon[31461]: pgmap v419: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:40.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 -- 172.21.15.97:0/3327765393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc106840 msgr2=0x7f5abc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:40.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 --2- 172.21.15.97:0/3327765393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc106840 0x7f5abc108c30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009990 tx=0x7f5aa8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 -- 172.21.15.97:0/3327765393 shutdown_connections 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 --2- 172.21.15.97:0/3327765393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc103f10 0x7f5abc106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 --2- 172.21.15.97:0/3327765393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc109170 0x7f5abc10b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 --2- 172.21.15.97:0/3327765393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc106840 0x7f5abc108c30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 -- 172.21.15.97:0/3327765393 >> 172.21.15.97:0/3327765393 conn(0x7f5abc0fd920 msgr2=0x7f5abc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 -- 172.21.15.97:0/3327765393 shutdown_connections 2024-04-26T21:32:40.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.344+0000 7f5ac4600700 1 -- 172.21.15.97:0/3327765393 wait complete. 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 Processor -- start 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 -- start start 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc103f10 0x7f5abc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc106840 0x7f5abc19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10ee30 con 0x7f5abc109170 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10ecb0 con 0x7f5abc106840 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.345+0000 7f5ac4600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10efb0 con 0x7f5abc103f10 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38348/0 (socket says 172.21.15.97:38348) 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 -- 172.21.15.97:0/1359714794 learned_addr learned my addr 172.21.15.97:0/1359714794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac1c00700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc103f10 0x7f5abc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac1200700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc106840 0x7f5abc19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 -- 172.21.15.97:0/1359714794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc103f10 msgr2=0x7f5abc19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc103f10 0x7f5abc19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 -- 172.21.15.97:0/1359714794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc106840 msgr2=0x7f5abc19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc106840 0x7f5abc19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.346+0000 7f5ac2600700 1 -- 172.21.15.97:0/1359714794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc1a33d0 con 0x7f5abc109170 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5ac2600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b4b0 tx=0x7f5aac00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac015020 con 0x7f5abc109170 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5aac011cc0 con 0x7f5abc109170 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac010c10 con 0x7f5abc109170 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5abc1a36c0 con 0x7f5abc109170 2024-04-26T21:32:40.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.347+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5abc1a3c30 con 0x7f5abc109170 2024-04-26T21:32:40.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.349+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5abc10acf0 con 0x7f5abc109170 2024-04-26T21:32:40.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.351+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5aac018020 con 0x7f5abc109170 2024-04-26T21:32:40.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.352+0000 7f5aba000700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 0x7f5aa00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.352+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5aac0d0080 con 0x7f5abc109170 2024-04-26T21:32:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.352+0000 7f5ac1c00700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 0x7f5aa00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.353+0000 7f5ac1c00700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 0x7f5aa00a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40060b0 tx=0x7f5ab400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.355+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5aac01f020 con 0x7f5abc109170 2024-04-26T21:32:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.601+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5abc063860 con 0x7f5aa00a3d50 2024-04-26T21:32:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.604+0000 7f5aba000700 1 -- 172.21.15.97:0/1359714794 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5abc063860 con 0x7f5aa00a3d50 2024-04-26T21:32:40.604 INFO:teuthology.orchestra.run.smithi097.stdout:Backend: cephadm 2024-04-26T21:32:40.604 INFO:teuthology.orchestra.run.smithi097.stdout:Available: Yes 2024-04-26T21:32:40.604 INFO:teuthology.orchestra.run.smithi097.stdout:Paused: No 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.606+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 msgr2=0x7f5aa00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.606+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 0x7f5aa00a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40060b0 tx=0x7f5ab400c040 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.606+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 msgr2=0x7f5abc1a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.606+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b4b0 tx=0x7f5aac00b7c0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 shutdown_connections 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5aa00a3d50 0x7f5aa00a6210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5abc109170 0x7f5abc1a2d30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5abc103f10 0x7f5abc19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 --2- 172.21.15.97:0/1359714794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5abc106840 0x7f5abc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 >> 172.21.15.97:0/1359714794 conn(0x7f5abc0fd920 msgr2=0x7f5abc1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 shutdown_connections 2024-04-26T21:32:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:40.607+0000 7f5ac4600700 1 -- 172.21.15.97:0/1359714794 wait complete. 2024-04-26T21:32:41.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T21:32:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:41.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:41 smithi097 ceph-mon[28127]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:41.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:41 smithi097 ceph-mon[28127]: pgmap v420: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:41 smithi111 ceph-mon[31001]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:41 smithi111 ceph-mon[31001]: pgmap v420: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:41 smithi143 ceph-mon[31461]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:42.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:41 smithi143 ceph-mon[31461]: pgmap v420: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 -- 172.21.15.97:0/1926259896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec109ac0 msgr2=0x7f16ec109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 --2- 172.21.15.97:0/1926259896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec109ac0 0x7f16ec109ee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0099c0 tx=0x7f16dc009cd0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 -- 172.21.15.97:0/1926259896 shutdown_connections 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 --2- 172.21.15.97:0/1926259896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec10b660 0x7f16ec111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 --2- 172.21.15.97:0/1926259896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec109ac0 0x7f16ec109ee0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 --2- 172.21.15.97:0/1926259896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10acc0 0x7f16ec10b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 -- 172.21.15.97:0/1926259896 >> 172.21.15.97:0/1926259896 conn(0x7f16ec078a90 msgr2=0x7f16ec07aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 -- 172.21.15.97:0/1926259896 shutdown_connections 2024-04-26T21:32:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.805+0000 7f16f2200700 1 -- 172.21.15.97:0/1926259896 wait complete. 2024-04-26T21:32:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.806+0000 7f16f2200700 1 Processor -- start 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.806+0000 7f16f2200700 1 -- start start 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec109ac0 0x7f16ec19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 0x7f16ec1a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec115450 con 0x7f16ec10acc0 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1152d0 con 0x7f16ec10b660 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16f2200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1155d0 con 0x7f16ec109ac0 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16eaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43872/0 (socket says 172.21.15.97:43872) 2024-04-26T21:32:43.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16eaa00700 1 -- 172.21.15.97:0/1653512342 learned_addr learned my addr 172.21.15.97:0/1653512342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.807+0000 7f16ebe00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 0x7f16ec1a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eb400700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec109ac0 0x7f16ec19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 -- 172.21.15.97:0/1653512342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec109ac0 msgr2=0x7f16ec19fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec109ac0 0x7f16ec19fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 -- 172.21.15.97:0/1653512342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 msgr2=0x7f16ec1a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 0x7f16ec1a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 -- 172.21.15.97:0/1653512342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec1a7780 con 0x7f16ec10acc0 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16ebe00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 0x7f16ec1a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:32:43.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16eaa00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f16d800d980 tx=0x7f16d800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8016070 con 0x7f16ec10acc0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16d8010460 con 0x7f16ec10acc0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec1a7a70 con 0x7f16ec10acc0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.808+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800f620 con 0x7f16ec10acc0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.809+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec1a7fb0 con 0x7f16ec10acc0 2024-04-26T21:32:43.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.810+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ec06a110 con 0x7f16ec10acc0 2024-04-26T21:32:43.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.812+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f16d8019070 con 0x7f16ec10acc0 2024-04-26T21:32:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.813+0000 7f16e3e00700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 0x7f16d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.813+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f16d80d0130 con 0x7f16ec10acc0 2024-04-26T21:32:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.813+0000 7f16eb400700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 0x7f16d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:43.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.814+0000 7f16eb400700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 0x7f16d00a6190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0098e0 tx=0x7f16dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:43.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:43.818+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16d809c9c0 con 0x7f16ec10acc0 2024-04-26T21:32:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.054+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f16ec063860 con 0x7f16d00a3cd0 2024-04-26T21:32:44.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:43 smithi097 ceph-mon[28127]: pgmap v421: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:44.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.095+0000 7f16e3e00700 1 -- 172.21.15.97:0/1653512342 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f16ec063860 con 0x7f16d00a3cd0 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.a smithi097 running (2m) 23s ago 2m 15.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 19bd127b4c70 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.b smithi111 running (117s) 44s ago 116s 24.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 2ff10f2c70fe 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.c smithi143 running (2m) 45s ago 2m 16.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 e32b9af47071 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.d smithi097 running (2m) 23s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 f0ba43200f48 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.e smithi111 running (2m) 44s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 21578beafa4e 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.f smithi143 running (2m) 45s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 cdfc7e6d0e29 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.g smithi097 running (2m) 23s ago 2m 16.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 27dc94437371 2024-04-26T21:32:44.097 INFO:teuthology.orchestra.run.smithi097.stdout:mds.h smithi111 running (2m) 44s ago 2m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 2e1384baef5b 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mds.i smithi143 running (2m) 45s ago 2m 16.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 a3f36c40fc85 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mds.j smithi097 running (2m) 23s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7ea633b498b8 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mds.k smithi111 running (2m) 44s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 45e7366b0eb8 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mds.l smithi143 running (2m) 45s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 0a58b5498d0c 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.x smithi097 *:9283,8765 running (12m) 23s ago 12m 516M - 19.0.0-3306-gbc612ffb 55dc54019ad8 75825ae1845a 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.y smithi111 *:8443,8765 running (9m) 44s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4fd235dbe821 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.z smithi143 *:8443,8765 running (9m) 45s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 53d47b99af6e 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a smithi097 running (12m) 23s ago 12m 50.4M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 8132cccb75ff 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b smithi111 running (10m) 44s ago 10m 39.3M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 7ebaf616a25d 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c smithi143 running (10m) 45s ago 10m 40.2M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 9148503ed2f4 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 smithi097 running (9m) 23s ago 9m 54.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 c21fc1dfc29e 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 smithi111 running (8m) 44s ago 8m 55.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5c66ff95ea0e 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 smithi143 running (8m) 45s ago 8m 56.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 6da82a3b014e 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 smithi097 running (7m) 23s ago 7m 46.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 182f5f469f2a 2024-04-26T21:32:44.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 smithi111 running (7m) 44s ago 7m 52.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 6cfe20ba6000 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 smithi143 running (6m) 45s ago 6m 53.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 416a9b639720 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 smithi097 running (5m) 23s ago 5m 51.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 f783cb106ab3 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 smithi111 running (5m) 44s ago 5m 45.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 035cae164ee8 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 smithi143 running (4m) 45s ago 4m 55.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 7d5e210e733f 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 smithi097 running (4m) 23s ago 4m 43.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5eeb2ef29d05 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 smithi111 running (3m) 44s ago 3m 44.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 847b002fca7d 2024-04-26T21:32:44.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 smithi143 running (3m) 45s ago 3m 50.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 3866a2de8841 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 msgr2=0x7f16d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 0x7f16d00a6190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0098e0 tx=0x7f16dc01a040 comp rx=0 tx=0).stop 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 msgr2=0x7f16ec1a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f16d800d980 tx=0x7f16d800dd40 comp rx=0 tx=0).stop 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 shutdown_connections 2024-04-26T21:32:44.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f16d00a3cd0 0x7f16d00a6190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16ec10b660 0x7f16ec1a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ec109ac0 0x7f16ec19fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 --2- 172.21.15.97:0/1653512342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ec10acc0 0x7f16ec1a0060 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 >> 172.21.15.97:0/1653512342 conn(0x7f16ec078a90 msgr2=0x7f16ec10e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 shutdown_connections 2024-04-26T21:32:44.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:44.099+0000 7f16f2200700 1 -- 172.21.15.97:0/1653512342 wait complete. 2024-04-26T21:32:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:43 smithi111 ceph-mon[31001]: pgmap v421: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:43 smithi143 ceph-mon[31461]: pgmap v421: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:44.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T21:32:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:46.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:45 smithi143 ceph-mon[31461]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:46.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:45 smithi143 ceph-mon[31461]: pgmap v422: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:45 smithi097 ceph-mon[28127]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:46.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:45 smithi097 ceph-mon[28127]: pgmap v422: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:45 smithi111 ceph-mon[31001]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:45 smithi111 ceph-mon[31001]: pgmap v422: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 -- 172.21.15.97:0/3241117208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48109160 msgr2=0x7fda4810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 --2- 172.21.15.97:0/3241117208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48109160 0x7fda4810b5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b0a0 tx=0x7fda4000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 -- 172.21.15.97:0/3241117208 shutdown_connections 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 --2- 172.21.15.97:0/3241117208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48103f00 0x7fda481062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 --2- 172.21.15.97:0/3241117208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48109160 0x7fda4810b5d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 --2- 172.21.15.97:0/3241117208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48106830 0x7fda48108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.439+0000 7fda4dc00700 1 -- 172.21.15.97:0/3241117208 >> 172.21.15.97:0/3241117208 conn(0x7fda480fd910 msgr2=0x7fda480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.440+0000 7fda4dc00700 1 -- 172.21.15.97:0/3241117208 shutdown_connections 2024-04-26T21:32:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.440+0000 7fda4dc00700 1 -- 172.21.15.97:0/3241117208 wait complete. 2024-04-26T21:32:47.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.440+0000 7fda4dc00700 1 Processor -- start 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 -- start start 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48103f00 0x7fda48199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 0x7fda481a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810edf0 con 0x7fda48106830 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810ec70 con 0x7fda48109160 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda4dc00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810ef70 con 0x7fda48103f00 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda46000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda46000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43886/0 (socket says 172.21.15.97:43886) 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.441+0000 7fda46000700 1 -- 172.21.15.97:0/2804253700 learned_addr learned my addr 172.21.15.97:0/2804253700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46a00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48103f00 0x7fda48199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda47400700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 0x7fda481a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 -- 172.21.15.97:0/2804253700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48103f00 msgr2=0x7fda48199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48103f00 0x7fda48199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 -- 172.21.15.97:0/2804253700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 msgr2=0x7fda481a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 0x7fda481a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 -- 172.21.15.97:0/2804253700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481a10a0 con 0x7fda48106830 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda47400700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 0x7fda481a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda46000700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d9b0 tx=0x7fda3400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34017070 con 0x7fda48106830 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda34004830 con 0x7fda48106830 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.442+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481a1390 con 0x7fda48106830 2024-04-26T21:32:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.443+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340053a0 con 0x7fda48106830 2024-04-26T21:32:47.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.443+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a1880 con 0x7fda48106830 2024-04-26T21:32:47.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.444+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4806a110 con 0x7fda48106830 2024-04-26T21:32:47.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.447+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda340049e0 con 0x7fda48106830 2024-04-26T21:32:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.448+0000 7fda3f400700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 0x7fda2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.448+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fda340d08b0 con 0x7fda48106830 2024-04-26T21:32:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.448+0000 7fda46a00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 0x7fda2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.449+0000 7fda46a00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 0x7fda2c0a6190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda38005ea0 tx=0x7fda38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.451+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda34020020 con 0x7fda48106830 2024-04-26T21:32:47.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.686+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fda48063860 con 0x7fda2c0a3cd0 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.697+0000 7fda3f400700 1 -- 172.21.15.97:0/2804253700 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fda48063860 con 0x7fda2c0a3cd0 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stdout:mds.all 12/12 49s ago 2m smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stdout:mgr 3/3 49s ago 9m smithi097=x;smithi111=y;smithi143=z;count:3 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stdout:mon 3/3 49s ago 10m smithi097:172.21.15.97=a;smithi111:172.21.15.111=b;smithi143:172.21.15.143=c;count:3 2024-04-26T21:32:47.698 INFO:teuthology.orchestra.run.smithi097.stdout:osd 12 49s ago - 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.699+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 msgr2=0x7fda2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.699+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 0x7fda2c0a6190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda38005ea0 tx=0x7fda38005e30 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 msgr2=0x7fda48199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d9b0 tx=0x7fda3400dd70 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 shutdown_connections 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fda2c0a3cd0 0x7fda2c0a6190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda48109160 0x7fda481a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda48103f00 0x7fda48199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 --2- 172.21.15.97:0/2804253700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda48106830 0x7fda48199980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 >> 172.21.15.97:0/2804253700 conn(0x7fda480fd910 msgr2=0x7fda481047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:47.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 shutdown_connections 2024-04-26T21:32:47.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:47.700+0000 7fda4dc00700 1 -- 172.21.15.97:0/2804253700 wait complete. 2024-04-26T21:32:48.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:47 smithi097 ceph-mon[28127]: pgmap v423: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:47 smithi111 ceph-mon[31001]: pgmap v423: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:47 smithi143 ceph-mon[31461]: pgmap v423: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:48.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T21:32:48.970 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:49.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:48 smithi097 ceph-mon[28127]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:48 smithi111 ceph-mon[31001]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:48 smithi143 ceph-mon[31461]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:49 smithi111 ceph-mon[31001]: pgmap v424: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:50.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:49 smithi097 ceph-mon[28127]: pgmap v424: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:50.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:49 smithi143 ceph-mon[31461]: pgmap v424: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:51.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 -- 172.21.15.97:0/2008122278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc105020 msgr2=0x7f9cdc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 --2- 172.21.15.97:0/2008122278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc105020 0x7f9cdc10b8d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400b0d0 tx=0x7f9cd400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 -- 172.21.15.97:0/2008122278 shutdown_connections 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 --2- 172.21.15.97:0/2008122278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc103480 0x7f9cdc1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 --2- 172.21.15.97:0/2008122278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc105020 0x7f9cdc10b8d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 --2- 172.21.15.97:0/2008122278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc104680 0x7f9cdc104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 -- 172.21.15.97:0/2008122278 >> 172.21.15.97:0/2008122278 conn(0x7f9cdc0fea20 msgr2=0x7f9cdc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 -- 172.21.15.97:0/2008122278 shutdown_connections 2024-04-26T21:32:51.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.214+0000 7f9ce3400700 1 -- 172.21.15.97:0/2008122278 wait complete. 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 Processor -- start 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 -- start start 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc103480 0x7f9cdc109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 0x7f9cdc105e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10ee00 con 0x7f9cdc104680 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10ec80 con 0x7f9cdc105020 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9ce3400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10ef80 con 0x7f9cdc103480 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9cdbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:51.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9cdbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47920/0 (socket says 172.21.15.97:47920) 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.216+0000 7f9cdbe00700 1 -- 172.21.15.97:0/781691440 learned_addr learned my addr 172.21.15.97:0/781691440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9ce0a00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc103480 0x7f9cdc109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9ce1400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 0x7f9cdc105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 -- 172.21.15.97:0/781691440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc103480 msgr2=0x7f9cdc109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc103480 0x7f9cdc109290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 -- 172.21.15.97:0/781691440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 msgr2=0x7f9cdc105e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 0x7f9cdc105e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 -- 172.21.15.97:0/781691440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cdc1066e0 con 0x7f9cdc104680 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9ce1400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 0x7f9cdc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:51.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cdbe00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c9b0 tx=0x7f9cc800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8007b00 con 0x7f9cdc104680 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cc8007ca0 con 0x7f9cdc104680 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.217+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cdc1a7ae0 con 0x7f9cdc104680 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.218+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8018660 con 0x7f9cdc104680 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.218+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cdc1a7f30 con 0x7f9cdc104680 2024-04-26T21:32:51.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.219+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cdc1a81c0 con 0x7f9cdc104680 2024-04-26T21:32:51.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.221+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9cc800f450 con 0x7f9cdc104680 2024-04-26T21:32:51.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.222+0000 7f9cd9600700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 0x7f9cc40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:51.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.223+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9cc80d05b0 con 0x7f9cdc104680 2024-04-26T21:32:51.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.223+0000 7f9ce0a00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 0x7f9cc40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:51.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.223+0000 7f9ce0a00700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 0x7f9cc40a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005fd0 tx=0x7f9cd0005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:51.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.227+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9cc8014030 con 0x7f9cdc104680 2024-04-26T21:32:51.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.459+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9cdc1a8390 con 0x7f9cc40a3d50 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.463+0000 7f9cd9600700 1 -- 172.21.15.97:0/781691440 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9cdc1a8390 con 0x7f9cc40a3d50 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stdout:HOST ADDR LABELS STATUS 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 172.21.15.97 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stdout:smithi111 172.21.15.111 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stdout:smithi143 172.21.15.143 2024-04-26T21:32:51.464 INFO:teuthology.orchestra.run.smithi097.stdout:3 hosts in cluster 2024-04-26T21:32:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 msgr2=0x7f9cc40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 0x7f9cc40a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005fd0 tx=0x7f9cd0005dd0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 msgr2=0x7f9cdc1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c9b0 tx=0x7f9cc800cd70 comp rx=0 tx=0).stop 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 shutdown_connections 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9cc40a3d50 0x7f9cc40a6210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cdc105020 0x7f9cdc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cdc103480 0x7f9cdc109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 --2- 172.21.15.97:0/781691440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cdc104680 0x7f9cdc1097d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 >> 172.21.15.97:0/781691440 conn(0x7f9cdc0fea20 msgr2=0x7f9cdc075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.466+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 shutdown_connections 2024-04-26T21:32:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:51.467+0000 7f9ce3400700 1 -- 172.21.15.97:0/781691440 wait complete. 2024-04-26T21:32:52.140 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:51 smithi097 ceph-mon[28127]: pgmap v425: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:51 smithi111 ceph-mon[31001]: pgmap v425: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:52.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T21:32:52.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:51 smithi143 ceph-mon[31461]: pgmap v425: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:52 smithi097 ceph-mon[28127]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:53.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:52 smithi097 ceph-mon[28127]: pgmap v426: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:52 smithi111 ceph-mon[31001]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:52 smithi111 ceph-mon[31001]: pgmap v426: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:53.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:52 smithi143 ceph-mon[31461]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:53.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:52 smithi143 ceph-mon[31461]: pgmap v426: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 -- 172.21.15.97:0/2777935096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf4104f50 msgr2=0x7f0bf410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 --2- 172.21.15.97:0/2777935096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf4104f50 0x7f0bf410b800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00b0d0 tx=0x7f0bec00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 -- 172.21.15.97:0/2777935096 shutdown_connections 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 --2- 172.21.15.97:0/2777935096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf4104f50 0x7f0bf410b800 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 --2- 172.21.15.97:0/2777935096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf41033b0 0x7f0bf41037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 --2- 172.21.15.97:0/2777935096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41045b0 0x7f0bf4104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 -- 172.21.15.97:0/2777935096 >> 172.21.15.97:0/2777935096 conn(0x7f0bf40fe950 msgr2=0x7f0bf4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 -- 172.21.15.97:0/2777935096 shutdown_connections 2024-04-26T21:32:54.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.720+0000 7f0bf9600700 1 -- 172.21.15.97:0/2777935096 wait complete. 2024-04-26T21:32:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.721+0000 7f0bf9600700 1 Processor -- start 2024-04-26T21:32:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 -- start start 2024-04-26T21:32:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41033b0 0x7f0bf4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 0x7f0bf41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410ed20 con 0x7f0bf41045b0 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410eba0 con 0x7f0bf41033b0 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf9600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410eea0 con 0x7f0bf4104f50 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47928/0 (socket says 172.21.15.97:47928) 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.722+0000 7f0bf3400700 1 -- 172.21.15.97:0/344210289 learned_addr learned my addr 172.21.15.97:0/344210289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3e00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41033b0 0x7f0bf4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf8c00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 0x7f0bf41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 -- 172.21.15.97:0/344210289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 msgr2=0x7f0bf41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 0x7f0bf41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 -- 172.21.15.97:0/344210289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41033b0 msgr2=0x7f0bf4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41033b0 0x7f0bf4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 -- 172.21.15.97:0/344210289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf41a0f60 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf8c00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 0x7f0bf41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf3400700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0be000d980 tx=0x7f0be000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0016070 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.723+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be0010460 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.724+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf41a1250 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.724+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be000f620 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.724+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a1790 con 0x7f0bf41045b0 2024-04-26T21:32:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.725+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf406a110 con 0x7f0bf41045b0 2024-04-26T21:32:54.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.727+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0be0019070 con 0x7f0bf41045b0 2024-04-26T21:32:54.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.728+0000 7f0bf0c00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 0x7f0bd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:54.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.728+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f0be00d0130 con 0x7f0bf41045b0 2024-04-26T21:32:54.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.728+0000 7f0bf3e00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 0x7f0bd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.729+0000 7f0bf3e00700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 0x7f0bd80a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0be4005fd0 tx=0x7f0be4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.731+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0be009cac0 con 0x7f0bf41045b0 2024-04-26T21:32:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.970+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0bf4063860 con 0x7f0bd80a3cd0 2024-04-26T21:32:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.974+0000 7f0bf0c00700 1 -- 172.21.15.97:0/344210289 <== mgr.14152 v2:172.21.15.97:6800/3172837618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0bf4063860 con 0x7f0bd80a3cd0 2024-04-26T21:32:54.975 INFO:teuthology.orchestra.run.smithi097.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stdout:smithi143 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330073400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 msgr2=0x7f0bd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 0x7f0bd80a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0be4005fd0 tx=0x7f0be4005e50 comp rx=0 tx=0).stop 2024-04-26T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 msgr2=0x7f0bf4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0be000d980 tx=0x7f0be000dd40 comp rx=0 tx=0).stop 2024-04-26T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 shutdown_connections 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f0bd80a3cd0 0x7f0bd80a6190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf41033b0 0x7f0bf4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf4104f50 0x7f0bf41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 --2- 172.21.15.97:0/344210289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf41045b0 0x7f0bf4199840 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 >> 172.21.15.97:0/344210289 conn(0x7f0bf40fe950 msgr2=0x7f0bf4107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.978+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 shutdown_connections 2024-04-26T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:54.979+0000 7f0bf9600700 1 -- 172.21.15.97:0/344210289 wait complete. 2024-04-26T21:32:55.826 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:32:55.836 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:32:55.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T21:32:56.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:55 smithi143 ceph-mon[31461]: pgmap v427: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:56.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:55 smithi097 ceph-mon[28127]: pgmap v427: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:55 smithi111 ceph-mon[31001]: pgmap v427: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:32:56.295 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:32:57.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:56 smithi143 ceph-mon[31461]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:57.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:56 smithi097 ceph-mon[28127]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:56 smithi111 ceph-mon[31001]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:32:57.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:57 smithi097 ceph-mon[28127]: pgmap v428: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:57 smithi143 ceph-mon[31461]: pgmap v428: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:57 smithi111 ceph-mon[31001]: pgmap v428: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 -- 172.21.15.97:0/2296332168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c1090a0 msgr2=0x7f9a3c10b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2296332168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c10b510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400b0a0 tx=0x7f9a3400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 -- 172.21.15.97:0/2296332168 shutdown_connections 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2296332168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c103e40 0x7f9a3c106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2296332168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c10b510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2296332168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c106770 0x7f9a3c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.422+0000 7f9a41a00700 1 -- 172.21.15.97:0/2296332168 >> 172.21.15.97:0/2296332168 conn(0x7f9a3c0fd830 msgr2=0x7f9a3c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.423+0000 7f9a41a00700 1 -- 172.21.15.97:0/2296332168 shutdown_connections 2024-04-26T21:32:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.423+0000 7f9a41a00700 1 -- 172.21.15.97:0/2296332168 wait complete. 2024-04-26T21:32:58.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 Processor -- start 2024-04-26T21:32:58.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 -- start start 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 0x7f9a3c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10edc0 con 0x7f9a3c103e40 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10ec40 con 0x7f9a3c106770 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a41a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3c10ef40 con 0x7f9a3c1090a0 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.424+0000 7f9a3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 -- 172.21.15.97:0/2706952066 learned_addr learned my addr 172.21.15.97:0/2706952066 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a33e00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 0x7f9a3c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a41000700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 -- 172.21.15.97:0/2706952066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 msgr2=0x7f9a3c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 -- 172.21.15.97:0/2706952066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 msgr2=0x7f9a3c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 0x7f9a3c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 -- 172.21.15.97:0/2706952066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3c1a10f0 con 0x7f9a3c103e40 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a33e00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 0x7f9a3c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a41000700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.425+0000 7f9a3be00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b560 tx=0x7f9a2c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.426+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c015020 con 0x7f9a3c103e40 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.426+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a2c011cc0 con 0x7f9a3c103e40 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.426+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3c101c90 con 0x7f9a3c103e40 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.426+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c010d30 con 0x7f9a3c103e40 2024-04-26T21:32:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.426+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3c102250 con 0x7f9a3c103e40 2024-04-26T21:32:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.427+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3c06a110 con 0x7f9a3c103e40 2024-04-26T21:32:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.429+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a2c018020 con 0x7f9a3c103e40 2024-04-26T21:32:58.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.431+0000 7f9a39600700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 0x7f9a200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:32:58.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.431+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f9a2c0cfc90 con 0x7f9a3c103e40 2024-04-26T21:32:58.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.431+0000 7f9a33e00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 0x7f9a200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:32:58.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.432+0000 7f9a33e00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 0x7f9a200a6190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007900 tx=0x7f9a24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:32:58.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.436+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9a2c01f020 con 0x7f9a3c103e40 2024-04-26T21:32:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.710+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9a3c0fed90 con 0x7f9a3c103e40 2024-04-26T21:32:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.711+0000 7f9a39600700 1 -- 172.21.15.97:0/2706952066 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3094 (secure 0 0 0) 0x7f9a2c09c630 con 0x7f9a3c103e40 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:e26 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:legacy client fscid: 1 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem 'cephfs' (1) 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:fs_name cephfs 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 26 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-04-26T21:30:53.245300+0000 2024-04-26T21:32:58.714 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-04-26T21:31:00.694971+0000 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:tableserver 0 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:root 0 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:session_timeout 300 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:session_autoclose 300 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:max_file_size 1099511627776 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:max_xattr_size 65536 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:required_client_features {} 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure 0 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure_osd_epoch 0 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:max_mds 5 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:in 0,1,2,3,4 2024-04-26T21:32:58.715 INFO:teuthology.orchestra.run.smithi097.stdout:up {0=34502,1=34460,2=34433,3=34400,4=24487} 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:failed 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:damaged 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:stopped 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:data_pools [3] 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:metadata_pool 2 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:inline_data disabled 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:balancer 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:bal_rank_mask -1 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:standby_count_wanted 0 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:qdb_cluster leader: 34502 members: 34400,24487,34433,34502,34460 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b{0:34502} state up:active seq 3 addr [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.143:6836/1658655529,v1:172.21.15.143:6837/1658655529] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.h{1:34460} state up:active seq 8 addr [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.716 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.143:6834/4086222297,v1:172.21.15.143:6835/4086222297] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.e{2:34433} state up:active seq 11 addr [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.l{2:24409} state up:standby-replay seq 1 addr [v2:172.21.15.143:6832/845773802,v1:172.21.15.143:6833/845773802] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.k{3:34400} state up:active seq 15 addr [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a{3:14655} state up:standby-replay seq 1 addr [v2:172.21.15.97:6840/1900627986,v1:172.21.15.97:6841/1900627986] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.i{4:24487} state up:active seq 8 addr [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.97:6838/3674554302,v1:172.21.15.97:6839/3674554302] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:Standby daemons: 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.97:6834/4203195871,v1:172.21.15.97:6835/4203195871] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.717 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.97:6836/847789147,v1:172.21.15.97:6837/847789147] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:32:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 msgr2=0x7f9a200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 0x7f9a200a6190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007900 tx=0x7f9a24008040 comp rx=0 tx=0).stop 2024-04-26T21:32:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 msgr2=0x7f9a3c199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:32:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b560 tx=0x7f9a2c00b920 comp rx=0 tx=0).stop 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 shutdown_connections 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9a200a3cd0 0x7f9a200a6190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3c1090a0 0x7f9a3c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a3c103e40 0x7f9a3c199490 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 --2- 172.21.15.97:0/2706952066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a3c106770 0x7f9a3c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 >> 172.21.15.97:0/2706952066 conn(0x7f9a3c0fd830 msgr2=0x7f9a3c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 shutdown_connections 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:32:58.716+0000 7f9a41a00700 1 -- 172.21.15.97:0/2706952066 wait complete. 2024-04-26T21:32:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:32:59.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:58 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2706952066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:32:59.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:58 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2706952066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:32:59.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:58 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2706952066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:32:59.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T21:33:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:33:00.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:32:59 smithi143 ceph-mon[31461]: pgmap v429: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:00.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:32:59 smithi097 ceph-mon[28127]: pgmap v429: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:32:59 smithi111 ceph-mon[31001]: pgmap v429: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:01.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:01 smithi097 ceph-mon[28127]: pgmap v430: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:02.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:01 smithi143 ceph-mon[31461]: pgmap v430: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 -- 172.21.15.97:0/3413334694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 msgr2=0x7fcadc076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 --2- 172.21.15.97:0/3413334694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc076280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009990 tx=0x7fcac4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 -- 172.21.15.97:0/3413334694 shutdown_connections 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 --2- 172.21.15.97:0/3413334694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc10b3b0 0x7fcadc10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 --2- 172.21.15.97:0/3413334694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc0773d0 0x7fcadc075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 --2- 172.21.15.97:0/3413334694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc076280 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.083+0000 7fcae1a00700 1 -- 172.21.15.97:0/3413334694 >> 172.21.15.97:0/3413334694 conn(0x7fcadc0fea70 msgr2=0x7fcadc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.084+0000 7fcae1a00700 1 -- 172.21.15.97:0/3413334694 shutdown_connections 2024-04-26T21:33:02.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.084+0000 7fcae1a00700 1 -- 172.21.15.97:0/3413334694 wait complete. 2024-04-26T21:33:02.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.085+0000 7fcae1a00700 1 Processor -- start 2024-04-26T21:33:02.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.085+0000 7fcae1a00700 1 -- start start 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc0773d0 0x7fcadc19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 0x7fcadc1a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc111030 con 0x7fcadc075e00 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc110eb0 con 0x7fcadc0773d0 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcae1a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc1111b0 con 0x7fcadc10b3b0 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcadaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcadaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44472/0 (socket says 172.21.15.97:44472) 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcadaa00700 1 -- 172.21.15.97:0/2570395894 learned_addr learned my addr 172.21.15.97:0/2570395894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcad2000700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc0773d0 0x7fcadc19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.086+0000 7fcadb400700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 0x7fcadc1a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:02.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 -- 172.21.15.97:0/2570395894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 msgr2=0x7fcadc1a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 0x7fcadc1a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 -- 172.21.15.97:0/2570395894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc0773d0 msgr2=0x7fcadc19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc0773d0 0x7fcadc19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 -- 172.21.15.97:0/2570395894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc1a3350 con 0x7fcadc075e00 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadb400700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 0x7fcadc1a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:02.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcadaa00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcacc007f40 tx=0x7fcacc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00dd50 con 0x7fcadc075e00 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcacc004bb0 con 0x7fcadc075e00 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc1a3640 con 0x7fcadc075e00 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0133f0 con 0x7fcadc075e00 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.087+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc1a3b80 con 0x7fcadc075e00 2024-04-26T21:33:02.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.089+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcadc06a110 con 0x7fcadc075e00 2024-04-26T21:33:02.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.091+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcacc01a020 con 0x7fcadc075e00 2024-04-26T21:33:02.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.092+0000 7fcad3e00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 0x7fcac00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:02.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.092+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fcacc0d0460 con 0x7fcadc075e00 2024-04-26T21:33:02.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.092+0000 7fcad2000700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 0x7fcac00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:02.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.093+0000 7fcad2000700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 0x7fcac00a6210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcac4003820 tx=0x7fcac4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:02.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.095+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcacc010070 con 0x7fcadc075e00 2024-04-26T21:33:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:01 smithi111 ceph-mon[31001]: pgmap v430: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:02.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.319+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fcadc064030 con 0x7fcadc075e00 2024-04-26T21:33:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.320+0000 7fcad3e00700 1 -- 172.21.15.97:0/2570395894 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5212 (secure 0 0 0) 0x7fcacc09ccf0 con 0x7fcadc075e00 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 90 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-04-26T21:20:14.183633+0000 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-04-26T21:31:21.811271+0000 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 26 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2024-04-26T21:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client luminous 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release squid 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 12 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6802/3890416768,v1:172.21.15.97:6803/3890416768] [v2:172.21.15.97:6804/3890416768,v1:172.21.15.97:6805/3890416768] exists,up b100e5ac-c305-4e0b-80d1-246e9107f082 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/4100839926,v1:172.21.15.111:6801/4100839926] [v2:172.21.15.111:6802/4100839926,v1:172.21.15.111:6803/4100839926] exists,up 2702d8b6-9f28-421d-b2d1-9c38a9900bbf 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6800/1100983067,v1:172.21.15.143:6801/1100983067] [v2:172.21.15.143:6802/1100983067,v1:172.21.15.143:6803/1100983067] exists,up 7059e770-39f5-4850-bfa3-521f33c394d0 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6810/92668618,v1:172.21.15.97:6811/92668618] [v2:172.21.15.97:6812/92668618,v1:172.21.15.97:6813/92668618] exists,up 5e8d302e-f0d1-4f1d-9464-905160e2eb4d 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/3555238271,v1:172.21.15.111:6809/3555238271] [v2:172.21.15.111:6810/3555238271,v1:172.21.15.111:6811/3555238271] exists,up 672a72b1-edd4-4392-b5ed-1167ae4c4205 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 40 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6808/1354252557,v1:172.21.15.143:6809/1354252557] [v2:172.21.15.143:6810/1354252557,v1:172.21.15.143:6811/1354252557] exists,up f52a82b9-6a36-4c0a-8287-f853589258c3 2024-04-26T21:33:02.324 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6818/3234116941,v1:172.21.15.97:6819/3234116941] [v2:172.21.15.97:6820/3234116941,v1:172.21.15.97:6821/3234116941] exists,up b3c68893-2e7a-4b3a-bbce-eb87ba907a25 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/3525126129,v1:172.21.15.111:6817/3525126129] [v2:172.21.15.111:6818/3525126129,v1:172.21.15.111:6819/3525126129] exists,up fb8c32d3-48dc-4e01-8e3b-127389176afd 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6816/2919334001,v1:172.21.15.143:6817/2919334001] [v2:172.21.15.143:6818/2919334001,v1:172.21.15.143:6819/2919334001] exists,up 5bd31888-d587-4094-a159-3c0f46dd71d2 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6826/3567005259,v1:172.21.15.97:6827/3567005259] [v2:172.21.15.97:6828/3567005259,v1:172.21.15.97:6829/3567005259] exists,up ff1702b9-db9d-48f5-835f-4ababb3bc2ed 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 up in weight 1 up_from 75 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/3431248583,v1:172.21.15.111:6825/3431248583] [v2:172.21.15.111:6826/3431248583,v1:172.21.15.111:6827/3431248583] exists,up db65ed02-3eb7-4c51-a807-bf051e380b87 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6824/921154932,v1:172.21.15.143:6825/921154932] [v2:172.21.15.143:6826/921154932,v1:172.21.15.143:6827/921154932] exists,up 17cd4c20-11c9-426a-9504-09d4fe119a5a 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.24 [4,9] 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.2c [4,9] 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.33 [5,9] 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/117675142 expires 2024-04-27T21:21:20.482343+0000 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/1403922799 expires 2024-04-27T21:21:20.482343+0000 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3008702290 expires 2024-04-27T21:20:46.262192+0000 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2383506754 expires 2024-04-27T21:21:20.482343+0000 2024-04-26T21:33:02.325 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/2998338527 expires 2024-04-27T21:20:46.262192+0000 2024-04-26T21:33:02.326 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/1403922799 expires 2024-04-27T21:21:20.482343+0000 2024-04-26T21:33:02.326 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3810497084 expires 2024-04-27T21:20:46.262192+0000 2024-04-26T21:33:02.326 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2341580572 expires 2024-04-27T21:21:20.482343+0000 2024-04-26T21:33:02.326 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3660896935 expires 2024-04-27T21:20:46.262192+0000 2024-04-26T21:33:02.326 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/2998338527 expires 2024-04-27T21:20:46.262192+0000 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 msgr2=0x7fcac00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 0x7fcac00a6210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcac4003820 tx=0x7fcac4005fd0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 msgr2=0x7fcadc19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcacc007f40 tx=0x7fcacc00d3b0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 shutdown_connections 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fcac00a3d50 0x7fcac00a6210 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcadc10b3b0 0x7fcadc1a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcadc075e00 0x7fcadc19b6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 --2- 172.21.15.97:0/2570395894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcadc0773d0 0x7fcadc19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 >> 172.21.15.97:0/2570395894 conn(0x7fcadc0fea70 msgr2=0x7fcadc107300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 shutdown_connections 2024-04-26T21:33:02.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:02.325+0000 7fcae1a00700 1 -- 172.21.15.97:0/2570395894 wait complete. 2024-04-26T21:33:03.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:02 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2570395894' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:33:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:02 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2570395894' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:33:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:02 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2570395894' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:33:03.189 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T21:33:03.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:33:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 -- 172.21.15.97:0/734213808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630105b50 msgr2=0x7f0630105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 --2- 172.21.15.97:0/734213808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630105b50 0x7f0630105fd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 -- 172.21.15.97:0/734213808 shutdown_connections 2024-04-26T21:33:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 --2- 172.21.15.97:0/734213808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630106510 0x7f063010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 --2- 172.21.15.97:0/734213808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630105b50 0x7f0630105fd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 --2- 172.21.15.97:0/734213808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f0630104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 -- 172.21.15.97:0/734213808 >> 172.21.15.97:0/734213808 conn(0x7f06300ffeb0 msgr2=0x7f0630102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 -- 172.21.15.97:0/734213808 shutdown_connections 2024-04-26T21:33:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.404+0000 7f062fe00640 1 -- 172.21.15.97:0/734213808 wait complete. 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.405+0000 7f062fe00640 1 Processor -- start 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.405+0000 7f062fe00640 1 -- start start 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.405+0000 7f062fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f063019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.405+0000 7f062fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630105b50 0x7f063019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06301103f0 con 0x7f0630106510 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0630110270 con 0x7f0630104960 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0630110570 con 0x7f0630105b50 2024-04-26T21:33:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44484/0 (socket says 172.21.15.97:44484) 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 -- 172.21.15.97:0/2573280148 learned_addr learned my addr 172.21.15.97:0/2573280148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062e000640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630105b50 0x7f063019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062ea00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f063019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 -- 172.21.15.97:0/2573280148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630105b50 msgr2=0x7f063019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630105b50 0x7f063019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 -- 172.21.15.97:0/2573280148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 msgr2=0x7f063019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f063019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062f400640 1 -- 172.21.15.97:0/2573280148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301a2750 con 0x7f0630106510 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.406+0000 7f062ea00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f063019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f062f400640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f062800b970 tx=0x7f062800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628019070 con 0x7f0630106510 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0628014440 con 0x7f0630106510 2024-04-26T21:33:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301a2a40 con 0x7f0630106510 2024-04-26T21:33:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280075d0 con 0x7f0630106510 2024-04-26T21:33:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.407+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301a2f90 con 0x7f0630106510 2024-04-26T21:33:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.408+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063010bcf0 con 0x7f0630106510 2024-04-26T21:33:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.410+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f062801c070 con 0x7f0630106510 2024-04-26T21:33:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.411+0000 7f0627400640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 0x7f06140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.412+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f06280cf9f0 con 0x7f0630106510 2024-04-26T21:33:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.412+0000 7f062ea00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 0x7f06140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.412+0000 7f062ea00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 0x7f06140a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06301059b0 tx=0x7f0620005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.416+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f062809c390 con 0x7f0630106510 2024-04-26T21:33:03.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.620+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f063002dcb0 con 0x7f0630106510 2024-04-26T21:33:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.623+0000 7f0627400640 1 -- 172.21.15.97:0/2573280148 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f062809bc90 con 0x7f0630106510 2024-04-26T21:33:03.624 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:33:03.624 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:33:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 msgr2=0x7f06140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 0x7f06140a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06301059b0 tx=0x7f0620005d30 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 msgr2=0x7f06301a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f062800b970 tx=0x7f062800be30 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 shutdown_connections 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f06140a3cc0 0x7f06140a6180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0630106510 0x7f06301a20e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.625+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0630105b50 0x7f063019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.626+0000 7f062fe00640 1 --2- 172.21.15.97:0/2573280148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0630104960 0x7f063019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.626+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 >> 172.21.15.97:0/2573280148 conn(0x7f06300ffeb0 msgr2=0x7f0630108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.626+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 shutdown_connections 2024-04-26T21:33:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.626+0000 7f062fe00640 1 -- 172.21.15.97:0/2573280148 wait complete. 2024-04-26T21:33:03.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:33:03.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 -- 172.21.15.97:0/2525874318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501023c0 msgr2=0x7f3850102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 --2- 172.21.15.97:0/2525874318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501023c0 0x7f3850102840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3848009ca0 tx=0x7f384802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 -- 172.21.15.97:0/2525874318 shutdown_connections 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 --2- 172.21.15.97:0/2525874318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501023c0 0x7f3850102840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 --2- 172.21.15.97:0/2525874318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3850101a20 0x7f3850101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 --2- 172.21.15.97:0/2525874318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38501010c0 0x7f38501014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 -- 172.21.15.97:0/2525874318 >> 172.21.15.97:0/2525874318 conn(0x7f38500fcca0 msgr2=0x7f38500ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 -- 172.21.15.97:0/2525874318 shutdown_connections 2024-04-26T21:33:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.850+0000 7f3856a00640 1 -- 172.21.15.97:0/2525874318 wait complete. 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 Processor -- start 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 -- start start 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3850101a20 0x7f385019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38501023c0 0x7f38501a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38501104c0 con 0x7f38501010c0 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3850110340 con 0x7f38501023c0 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.851+0000 7f3856a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3850110640 con 0x7f3850101a20 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44500/0 (socket says 172.21.15.97:44500) 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 -- 172.21.15.97:0/2835744844 learned_addr learned my addr 172.21.15.97:0/2835744844 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3854c00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3850101a20 0x7f385019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 -- 172.21.15.97:0/2835744844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3850101a20 msgr2=0x7f385019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3850101a20 0x7f385019b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 -- 172.21.15.97:0/2835744844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38501023c0 msgr2=0x7f38501a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3856000640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38501023c0 0x7f38501a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38501023c0 0x7f38501a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 -- 172.21.15.97:0/2835744844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501a27b0 con 0x7f38501010c0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.852+0000 7f3855600640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f384400b4a0 tx=0x7f384400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.853+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844014070 con 0x7f38501010c0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.853+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3844010aa0 con 0x7f38501010c0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.853+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844017b70 con 0x7f38501010c0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.853+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38501a2aa0 con 0x7f38501010c0 2024-04-26T21:33:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.853+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a2fa0 con 0x7f38501010c0 2024-04-26T21:33:03.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.854+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385010be10 con 0x7f38501010c0 2024-04-26T21:33:03.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.856+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3844017070 con 0x7f38501010c0 2024-04-26T21:33:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.857+0000 7f384e000640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 0x7f38380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.858+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f38440cf7d0 con 0x7f38501010c0 2024-04-26T21:33:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.858+0000 7f3854c00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 0x7f38380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:03.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.858+0000 7f3854c00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 0x7f38380a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f385019c0a0 tx=0x7f383c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:03.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:03.862+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f384409c270 con 0x7f38501010c0 2024-04-26T21:33:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:03 smithi097 ceph-mon[28127]: pgmap v431: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:04.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:03 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2573280148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.064+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f385002dd40 con 0x7f38501010c0 2024-04-26T21:33:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.065+0000 7f384e000640 1 -- 172.21.15.97:0/2835744844 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f384409bb70 con 0x7f38501010c0 2024-04-26T21:33:04.066 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 msgr2=0x7f38380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 0x7f38380a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f385019c0a0 tx=0x7f383c005c60 comp rx=0 tx=0).stop 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 msgr2=0x7f385019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f384400b4a0 tx=0x7f384400b960 comp rx=0 tx=0).stop 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 shutdown_connections 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38501023c0 0x7f38501a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3850101a20 0x7f385019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f38380a3c40 0x7f38380a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 --2- 172.21.15.97:0/2835744844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38501010c0 0x7f385019ab90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 >> 172.21.15.97:0/2835744844 conn(0x7f38500fcca0 msgr2=0x7f3850100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 shutdown_connections 2024-04-26T21:33:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.068+0000 7f3856a00640 1 -- 172.21.15.97:0/2835744844 wait complete. 2024-04-26T21:33:04.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:33:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:03 smithi111 ceph-mon[31001]: pgmap v431: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:03 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2573280148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:04.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:03 smithi143 ceph-mon[31461]: pgmap v431: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:04.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:03 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2573280148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 -- 172.21.15.97:0/3884619420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0102940 msgr2=0x7fe6e0102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/3884619420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0102940 0x7fe6e0102dc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009900 tx=0x7fe6cc02a410 comp rx=0 tx=0).stop 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 -- 172.21.15.97:0/3884619420 shutdown_connections 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/3884619420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0103300 0x7fe6e0109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/3884619420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0102940 0x7fe6e0102dc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/3884619420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 0x7fe6e0101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 -- 172.21.15.97:0/3884619420 >> 172.21.15.97:0/3884619420 conn(0x7fe6e00fcce0 msgr2=0x7fe6e00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 -- 172.21.15.97:0/3884619420 shutdown_connections 2024-04-26T21:33:04.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.300+0000 7fe6e6a00640 1 -- 172.21.15.97:0/3884619420 wait complete. 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.301+0000 7fe6e6a00640 1 Processor -- start 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.301+0000 7fe6e6a00640 1 -- start start 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 0x7fe6e019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0102940 0x7fe6e019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e2b0 con 0x7fe6e0103300 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e130 con 0x7fe6e0102940 2024-04-26T21:33:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e430 con 0x7fe6e0101750 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44520/0 (socket says 172.21.15.97:44520) 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 -- 172.21.15.97:0/1928719241 learned_addr learned my addr 172.21.15.97:0/1928719241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e4c00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0102940 0x7fe6e019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e5600640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 0x7fe6e019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 -- 172.21.15.97:0/1928719241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 msgr2=0x7fe6e019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 0x7fe6e019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 -- 172.21.15.97:0/1928719241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0102940 msgr2=0x7fe6e019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0102940 0x7fe6e019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.302+0000 7fe6e6000640 1 -- 172.21.15.97:0/1928719241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e01a27c0 con 0x7fe6e0103300 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6e6000640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800dac0 tx=0x7fe6d800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8017070 con 0x7fe6e0103300 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6d8003e50 con 0x7fe6e0103300 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e01a2ab0 con 0x7fe6e0103300 2024-04-26T21:33:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8010460 con 0x7fe6e0103300 2024-04-26T21:33:04.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.303+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01a2fd0 con 0x7fe6e0103300 2024-04-26T21:33:04.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.304+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e01a31a0 con 0x7fe6e0103300 2024-04-26T21:33:04.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.306+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6d80039c0 con 0x7fe6e0103300 2024-04-26T21:33:04.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.308+0000 7fe6de000640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 0x7fe6c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.308+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fe6d80d07a0 con 0x7fe6e0103300 2024-04-26T21:33:04.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.308+0000 7fe6e5600640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 0x7fe6c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.309+0000 7fe6e5600640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 0x7fe6c80a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.312+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe6d8020020 con 0x7fe6e0103300 2024-04-26T21:33:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.513+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe6e01a3530 con 0x7fe6e0103300 2024-04-26T21:33:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.514+0000 7fe6de000640 1 -- 172.21.15.97:0/1928719241 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe6d809d130 con 0x7fe6e0103300 2024-04-26T21:33:04.515 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:33:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 msgr2=0x7fe6c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 0x7fe6c80a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d4005850 comp rx=0 tx=0).stop 2024-04-26T21:33:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 msgr2=0x7fe6e01a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800dac0 tx=0x7fe6d800df80 comp rx=0 tx=0).stop 2024-04-26T21:33:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 shutdown_connections 2024-04-26T21:33:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe6c80a3c40 0x7fe6c80a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6e0103300 0x7fe6e01a2150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e0102940 0x7fe6e019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 --2- 172.21.15.97:0/1928719241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6e0101750 0x7fe6e019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 >> 172.21.15.97:0/1928719241 conn(0x7fe6e00fcce0 msgr2=0x7fe6e010a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 shutdown_connections 2024-04-26T21:33:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.517+0000 7fe6e6a00640 1 -- 172.21.15.97:0/1928719241 wait complete. 2024-04-26T21:33:04.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:33:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 -- 172.21.15.97:0/292490916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004102980 msgr2=0x7f2004109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 --2- 172.21.15.97:0/292490916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004102980 0x7f2004109210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009ca0 tx=0x7f1ffc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 -- 172.21.15.97:0/292490916 shutdown_connections 2024-04-26T21:33:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 --2- 172.21.15.97:0/292490916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004102980 0x7f2004109210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 --2- 172.21.15.97:0/292490916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004101fc0 0x7f2004102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 --2- 172.21.15.97:0/292490916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004100dd0 0x7f20041011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 -- 172.21.15.97:0/292490916 >> 172.21.15.97:0/292490916 conn(0x7f20040fc360 msgr2=0x7f20040fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 -- 172.21.15.97:0/292490916 shutdown_connections 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.745+0000 7f2008a00640 1 -- 172.21.15.97:0/292490916 wait complete. 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 Processor -- start 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 -- start start 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004101fc0 0x7f200419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004102980 0x7f20041a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f200410e1e0 con 0x7f2004100dd0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f200410e060 con 0x7f2004102980 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.746+0000 7f2008a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f200410e360 con 0x7f2004101fc0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44528/0 (socket says 172.21.15.97:44528) 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 -- 172.21.15.97:0/2292857728 learned_addr learned my addr 172.21.15.97:0/2292857728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2002a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004101fc0 0x7f200419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003e00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004102980 0x7f20041a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 -- 172.21.15.97:0/2292857728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004101fc0 msgr2=0x7f200419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004101fc0 0x7f200419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 -- 172.21.15.97:0/2292857728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004102980 msgr2=0x7f20041a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004102980 0x7f20041a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 -- 172.21.15.97:0/2292857728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041a2620 con 0x7f2004100dd0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.747+0000 7f2003400640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400d950 tx=0x7f1ff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.748+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4017070 con 0x7f2004100dd0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.748+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ff4009c30 con 0x7f2004100dd0 2024-04-26T21:33:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.748+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a2910 con 0x7f2004100dd0 2024-04-26T21:33:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.748+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400f620 con 0x7f2004100dd0 2024-04-26T21:33:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.748+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a2e10 con 0x7f2004100dd0 2024-04-26T21:33:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.749+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20041a3040 con 0x7f2004100dd0 2024-04-26T21:33:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.751+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ff401a070 con 0x7f2004100dd0 2024-04-26T21:33:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.752+0000 7f1ffbe00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 0x7f1fe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.753+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f1ff40d0090 con 0x7f2004100dd0 2024-04-26T21:33:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.753+0000 7f2002a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 0x7f1fe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.753+0000 7f2002a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 0x7f1fe80a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005fd0 tx=0x7f1ff0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.755+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ff4009de0 con 0x7f2004100dd0 2024-04-26T21:33:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.961+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20041a3210 con 0x7f2004100dd0 2024-04-26T21:33:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.962+0000 7f1ffbe00640 1 -- 172.21.15.97:0/2292857728 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f1ff409c920 con 0x7f2004100dd0 2024-04-26T21:33:04.963 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 msgr2=0x7f1fe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 0x7f1fe80a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005fd0 tx=0x7f1ff0005850 comp rx=0 tx=0).stop 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 msgr2=0x7f200419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400d950 tx=0x7f1ff400de10 comp rx=0 tx=0).stop 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 shutdown_connections 2024-04-26T21:33:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2004102980 0x7f20041a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2004101fc0 0x7f200419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f1fe80a3c40 0x7f1fe80a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 --2- 172.21.15.97:0/2292857728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2004100dd0 0x7f200419aa00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 >> 172.21.15.97:0/2292857728 conn(0x7f20040fc360 msgr2=0x7f200410a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 shutdown_connections 2024-04-26T21:33:04.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:04.965+0000 7f2008a00640 1 -- 172.21.15.97:0/2292857728 wait complete. 2024-04-26T21:33:04.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:33:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:04 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2835744844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:04 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1928719241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 -- 172.21.15.97:0/2480518213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 msgr2=0x7f86e0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 --2- 172.21.15.97:0/2480518213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e0104db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009900 tx=0x7f86d002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 -- 172.21.15.97:0/2480518213 shutdown_connections 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 --2- 172.21.15.97:0/2480518213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 --2- 172.21.15.97:0/2480518213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 0x7f86e0106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 --2- 172.21.15.97:0/2480518213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e0104db0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 -- 172.21.15.97:0/2480518213 >> 172.21.15.97:0/2480518213 conn(0x7f86e00fff00 msgr2=0x7f86e0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 -- 172.21.15.97:0/2480518213 shutdown_connections 2024-04-26T21:33:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.181+0000 7f86e4e00640 1 -- 172.21.15.97:0/2480518213 wait complete. 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 Processor -- start 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 -- start start 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 0x7f86e019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e01a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86e01103c0 con 0x7f86e0104990 2024-04-26T21:33:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0110240 con 0x7f86e0105b80 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.182+0000 7f86e4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0110540 con 0x7f86e0106540 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44546/0 (socket says 172.21.15.97:44546) 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 -- 172.21.15.97:0/451901924 learned_addr learned my addr 172.21.15.97:0/451901924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86dfe00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e01a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 -- 172.21.15.97:0/451901924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 msgr2=0x7f86e01a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e01a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86d7e00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 0x7f86e019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 -- 172.21.15.97:0/451901924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 msgr2=0x7f86e019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 0x7f86e019afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 -- 172.21.15.97:0/451901924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e01a2690 con 0x7f86e0104990 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86dfe00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e01a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.183+0000 7f86df400640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86d0031460 tx=0x7f86d0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.184+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d003f070 con 0x7f86e0104990 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.184+0000 7f86e4e00640 1 -- 172.21.15.97:0/451901924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e01a2920 con 0x7f86e0104990 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.184+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d0031da0 con 0x7f86e0104990 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.184+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0049060 con 0x7f86e0104990 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.184+0000 7f86e4e00640 1 -- 172.21.15.97:0/451901924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01a2e60 con 0x7f86e0104990 2024-04-26T21:33:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.185+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0063530 con 0x7f86e0104990 2024-04-26T21:33:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.187+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86d0042070 con 0x7f86e0104990 2024-04-26T21:33:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.188+0000 7f86dcc00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.189+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f86d00f1340 con 0x7f86e0104990 2024-04-26T21:33:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.189+0000 7f86d7e00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.189+0000 7f86d7e00640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 0x7f86c40a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86e019bf80 tx=0x7f86c8005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:05.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.193+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f86d00bdbd0 con 0x7f86e0104990 2024-04-26T21:33:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:04 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2835744844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:04 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1928719241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:04 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2835744844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:04 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1928719241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:05.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.431+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86e0063700 con 0x7f86e0104990 2024-04-26T21:33:05.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.433+0000 7f86dcc00640 1 -- 172.21.15.97:0/451901924 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7f86d00bd4d0 con 0x7f86e0104990 2024-04-26T21:33:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:33:05.434 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:33:05.434 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:33:05.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 msgr2=0x7f86c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 0x7f86c40a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86e019bf80 tx=0x7f86c8005cc0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 msgr2=0x7f86e019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86d0031460 tx=0x7f86d0003d50 comp rx=0 tx=0).stop 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 shutdown_connections 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86e0106540 0x7f86e01a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e0105b80 0x7f86e019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 --2- 172.21.15.97:0/451901924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e0104990 0x7f86e019aa70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 >> 172.21.15.97:0/451901924 conn(0x7f86e00fff00 msgr2=0x7f86e0108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.435+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 shutdown_connections 2024-04-26T21:33:05.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.436+0000 7f86d5600640 1 -- 172.21.15.97:0/451901924 wait complete. 2024-04-26T21:33:05.455 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:33:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 -- 172.21.15.97:0/3016730369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 msgr2=0x7f25a410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 --2- 172.21.15.97:0/3016730369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a410a600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2590009900 tx=0x7f259002a410 comp rx=0 tx=0).stop 2024-04-26T21:33:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 -- 172.21.15.97:0/3016730369 shutdown_connections 2024-04-26T21:33:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 --2- 172.21.15.97:0/3016730369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a410ab40 0x7f25a41113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 --2- 172.21.15.97:0/3016730369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a410a600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 --2- 172.21.15.97:0/3016730369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 -- 172.21.15.97:0/3016730369 >> 172.21.15.97:0/3016730369 conn(0x7f25a4077f40 msgr2=0x7f25a407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:05.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 -- 172.21.15.97:0/3016730369 shutdown_connections 2024-04-26T21:33:05.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.648+0000 7f25abc00640 1 -- 172.21.15.97:0/3016730369 wait complete. 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 Processor -- start 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 -- start start 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a410ab40 0x7f25a41a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41149d0 con 0x7f25a410a180 2024-04-26T21:33:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4114850 con 0x7f25a410ab40 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.649+0000 7f25abc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4114b50 con 0x7f25a4108f90 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44570/0 (socket says 172.21.15.97:44570) 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 -- 172.21.15.97:0/3530440323 learned_addr learned my addr 172.21.15.97:0/3530440323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a9200640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 -- 172.21.15.97:0/3530440323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 msgr2=0x7f25a419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 -- 172.21.15.97:0/3530440323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a410ab40 msgr2=0x7f25a41a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a410ab40 0x7f25a41a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 -- 172.21.15.97:0/3530440323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a41a6cd0 con 0x7f25a410a180 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a9200640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.650+0000 7f25a3e00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2590031460 tx=0x7f2590003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.651+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259003f070 con 0x7f25a410a180 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.651+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2590031da0 con 0x7f25a410a180 2024-04-26T21:33:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.651+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a41a6f60 con 0x7f25a410a180 2024-04-26T21:33:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.651+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590035030 con 0x7f25a410a180 2024-04-26T21:33:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.651+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a41a73c0 con 0x7f25a410a180 2024-04-26T21:33:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.652+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a4110330 con 0x7f25a410a180 2024-04-26T21:33:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.654+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2590042070 con 0x7f25a410a180 2024-04-26T21:33:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.656+0000 7f25a1600640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 0x7f25880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.656+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f25900f0f20 con 0x7f25a410a180 2024-04-26T21:33:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.656+0000 7f25a9200640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 0x7f25880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.657+0000 7f25a9200640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 0x7f25880a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25a4109fe0 tx=0x7f2594005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.661+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f25900bd7b0 con 0x7f25a410a180 2024-04-26T21:33:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.863+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25a402dd20 con 0x7f25a410a180 2024-04-26T21:33:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.866+0000 7f25a1600640 1 -- 172.21.15.97:0/3530440323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f25900bd0b0 con 0x7f25a410a180 2024-04-26T21:33:05.868 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:33:05.868 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 msgr2=0x7f25880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 0x7f25880a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25a4109fe0 tx=0x7f2594005e70 comp rx=0 tx=0).stop 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 msgr2=0x7f25a419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2590031460 tx=0x7f2590003c80 comp rx=0 tx=0).stop 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 shutdown_connections 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a410ab40 0x7f25a41a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f25880a3c40 0x7f25880a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a410a180 0x7f25a419f5f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 --2- 172.21.15.97:0/3530440323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a4108f90 0x7f25a419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 >> 172.21.15.97:0/3530440323 conn(0x7f25a4077f40 msgr2=0x7f25a410d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 shutdown_connections 2024-04-26T21:33:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:05.869+0000 7f25abc00640 1 -- 172.21.15.97:0/3530440323 wait complete. 2024-04-26T21:33:05.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:33:05.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:33:06.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 -- 172.21.15.97:0/2980503989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4200104470 msgr2=0x7f420010ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 --2- 172.21.15.97:0/2980503989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4200104470 0x7f420010ad00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009ca0 tx=0x7f41f802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 -- 172.21.15.97:0/2980503989 shutdown_connections 2024-04-26T21:33:06.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 --2- 172.21.15.97:0/2980503989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4200104470 0x7f420010ad00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 --2- 172.21.15.97:0/2980503989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 0x7f4200103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 --2- 172.21.15.97:0/2980503989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42001028c0 0x7f4200102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 -- 172.21.15.97:0/2980503989 >> 172.21.15.97:0/2980503989 conn(0x7f42000fde30 msgr2=0x7f4200100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 -- 172.21.15.97:0/2980503989 shutdown_connections 2024-04-26T21:33:06.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.095+0000 7f4206a00640 1 -- 172.21.15.97:0/2980503989 wait complete. 2024-04-26T21:33:06.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.096+0000 7f4206a00640 1 Processor -- start 2024-04-26T21:33:06.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.096+0000 7f4206a00640 1 -- start start 2024-04-26T21:33:06.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 0x7f420019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4200104470 0x7f42001a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f420010e260 con 0x7f42001028c0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f420010e0e0 con 0x7f4200104470 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f420010e3e0 con 0x7f4200103ab0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44592/0 (socket says 172.21.15.97:44592) 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 -- 172.21.15.97:0/1240916849 learned_addr learned my addr 172.21.15.97:0/1240916849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4204c00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 0x7f420019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 -- 172.21.15.97:0/1240916849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 msgr2=0x7f420019b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 0x7f420019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 -- 172.21.15.97:0/1240916849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4200104470 msgr2=0x7f42001a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4206000640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4200104470 0x7f42001a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4200104470 0x7f42001a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.097+0000 7f4205600640 1 -- 172.21.15.97:0/1240916849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001a2760 con 0x7f42001028c0 2024-04-26T21:33:06.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f4205600640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f41f000d770 tx=0x7f41f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0017070 con 0x7f42001028c0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41f0009c30 con 0x7f42001028c0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0003c50 con 0x7f42001028c0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a2a50 con 0x7f42001028c0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.098+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a2f50 con 0x7f42001028c0 2024-04-26T21:33:06.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.099+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42001a3180 con 0x7f42001028c0 2024-04-26T21:33:06.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.103+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41f0003df0 con 0x7f42001028c0 2024-04-26T21:33:06.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.104+0000 7f41fe000640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 0x7f41e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.104+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f41f00d02d0 con 0x7f42001028c0 2024-04-26T21:33:06.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.104+0000 7f4204c00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 0x7f41e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.105+0000 7f4204c00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 0x7f41e40a6180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005fd0 tx=0x7f41ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.107+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f41f0009de0 con 0x7f42001028c0 2024-04-26T21:33:06.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:05 smithi111 ceph-mon[31001]: pgmap v432: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:06.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:05 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2292857728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:06.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:05 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/451901924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:06.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:05 smithi143 ceph-mon[31461]: pgmap v432: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:06.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:05 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2292857728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:06.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:05 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/451901924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:06.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.307+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42001a3470 con 0x7f42001028c0 2024-04-26T21:33:06.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.308+0000 7f41fe000640 1 -- 172.21.15.97:0/1240916849 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f41f009cce0 con 0x7f42001028c0 2024-04-26T21:33:06.309 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:33:06.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.310+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 msgr2=0x7f41e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.310+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 0x7f41e40a6180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005fd0 tx=0x7f41ec005850 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.310+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 msgr2=0x7f420019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.310+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f41f000d770 tx=0x7f41f000dc30 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 shutdown_connections 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4200104470 0x7f42001a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4200103ab0 0x7f420019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f41e40a3cc0 0x7f41e40a6180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 --2- 172.21.15.97:0/1240916849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001028c0 0x7f420019ab40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 >> 172.21.15.97:0/1240916849 conn(0x7f42000fde30 msgr2=0x7f4200109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 shutdown_connections 2024-04-26T21:33:06.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.311+0000 7f4206a00640 1 -- 172.21.15.97:0/1240916849 wait complete. 2024-04-26T21:33:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:05 smithi097 ceph-mon[28127]: pgmap v432: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:05 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2292857728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:05 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/451901924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:06.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:33:06.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 -- 172.21.15.97:0/1604539502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4106620 msgr2=0x7ff1c410ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 --2- 172.21.15.97:0/1604539502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4106620 0x7ff1c410ceb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ca0 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 -- 172.21.15.97:0/1604539502 shutdown_connections 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 --2- 172.21.15.97:0/1604539502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4106620 0x7ff1c410ceb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 --2- 172.21.15.97:0/1604539502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1c4105c60 0x7ff1c41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 --2- 172.21.15.97:0/1604539502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1c4104a70 0x7ff1c4104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 -- 172.21.15.97:0/1604539502 >> 172.21.15.97:0/1604539502 conn(0x7ff1c40fffe0 msgr2=0x7ff1c4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 -- 172.21.15.97:0/1604539502 shutdown_connections 2024-04-26T21:33:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.531+0000 7ff1c3200640 1 -- 172.21.15.97:0/1604539502 wait complete. 2024-04-26T21:33:06.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.532+0000 7ff1c3200640 1 Processor -- start 2024-04-26T21:33:06.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.532+0000 7ff1c3200640 1 -- start start 2024-04-26T21:33:06.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1c4105c60 0x7ff1c419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1c4106620 0x7ff1c41a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4110480 con 0x7ff1c4104a70 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4110300 con 0x7ff1c4105c60 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4110600 con 0x7ff1c4106620 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44610/0 (socket says 172.21.15.97:44610) 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 -- 172.21.15.97:0/3723099478 learned_addr learned my addr 172.21.15.97:0/3723099478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 -- 172.21.15.97:0/3723099478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1c4106620 msgr2=0x7ff1c41a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1c4106620 0x7ff1c41a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 -- 172.21.15.97:0/3723099478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1c4105c60 msgr2=0x7ff1c419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1c4105c60 0x7ff1c419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 -- 172.21.15.97:0/3723099478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c41a2710 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.533+0000 7ff1c1e00640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c970 tx=0x7ff1b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.534+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4007c10 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.534+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1b4007db0 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.534+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c41a2a00 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.534+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400f7d0 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.534+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41a2f50 con 0x7ff1c4104a70 2024-04-26T21:33:06.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.535+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c410bd90 con 0x7ff1c4104a70 2024-04-26T21:33:06.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.537+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1b4018480 con 0x7ff1c4104a70 2024-04-26T21:33:06.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.538+0000 7ff1baa00640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 0x7ff1a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.538+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff1b40d08a0 con 0x7ff1c4104a70 2024-04-26T21:33:06.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.539+0000 7ff1c1400640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 0x7ff1a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.539+0000 7ff1c1400640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 0x7ff1a80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0005fd0 tx=0x7ff1b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.542+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff1b409d2c0 con 0x7ff1c4104a70 2024-04-26T21:33:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.745+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1c402dd40 con 0x7ff1c4104a70 2024-04-26T21:33:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.746+0000 7ff1baa00640 1 -- 172.21.15.97:0/3723099478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7ff1b409cbc0 con 0x7ff1c4104a70 2024-04-26T21:33:06.747 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.748+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 msgr2=0x7ff1a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.748+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 0x7ff1a80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0005fd0 tx=0x7ff1b0005850 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.748+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 msgr2=0x7ff1c419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.748+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c970 tx=0x7ff1b400ce30 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 shutdown_connections 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1c4106620 0x7ff1c41a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1c4105c60 0x7ff1c419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff1a80a3c40 0x7ff1a80a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 --2- 172.21.15.97:0/3723099478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c4104a70 0x7ff1c419aaf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 >> 172.21.15.97:0/3723099478 conn(0x7ff1c40fffe0 msgr2=0x7ff1c4108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 shutdown_connections 2024-04-26T21:33:06.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.749+0000 7ff1c3200640 1 -- 172.21.15.97:0/3723099478 wait complete. 2024-04-26T21:33:06.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:33:06.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 -- 172.21.15.97:0/3432495531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 msgr2=0x7f3a54103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 --2- 172.21.15.97:0/3432495531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a54103420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009900 tx=0x7f3a4402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 -- 172.21.15.97:0/3432495531 shutdown_connections 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 --2- 172.21.15.97:0/3432495531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 0x7f3a54106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 --2- 172.21.15.97:0/3432495531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 0x7f3a54105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 --2- 172.21.15.97:0/3432495531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a54103420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.972+0000 7f3a58c00640 1 -- 172.21.15.97:0/3432495531 >> 172.21.15.97:0/3432495531 conn(0x7f3a540fcbf0 msgr2=0x7f3a540ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.973+0000 7f3a58c00640 1 -- 172.21.15.97:0/3432495531 shutdown_connections 2024-04-26T21:33:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.973+0000 7f3a58c00640 1 -- 172.21.15.97:0/3432495531 wait complete. 2024-04-26T21:33:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 Processor -- start 2024-04-26T21:33:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 -- start start 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 0x7f3a5419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 0x7f3a541a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54114730 con 0x7f3a54100ff0 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3a541145b0 con 0x7f3a54106310 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a58c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3a541148b0 con 0x7f3a54103960 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44626/0 (socket says 172.21.15.97:44626) 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.974+0000 7f3a53400640 1 -- 172.21.15.97:0/1313927937 learned_addr learned my addr 172.21.15.97:0/1313927937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a4be00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 0x7f3a5419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53e00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 0x7f3a541a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 -- 172.21.15.97:0/1313927937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 msgr2=0x7f3a5419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 0x7f3a5419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 -- 172.21.15.97:0/1313927937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 msgr2=0x7f3a541a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 0x7f3a541a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 -- 172.21.15.97:0/1313927937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a541a6a40 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.975+0000 7f3a53400640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009230 tx=0x7f3a440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.976+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4403f030 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.976+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a44003c90 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.976+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a541a6cd0 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.976+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44035070 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.976+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a541a71c0 con 0x7f3a54100ff0 2024-04-26T21:33:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.977+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a540fe6e0 con 0x7f3a54100ff0 2024-04-26T21:33:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.979+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a44042030 con 0x7f3a54100ff0 2024-04-26T21:33:06.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.980+0000 7f3a50c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 0x7f3a380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:06.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.980+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3a440f0de0 con 0x7f3a54100ff0 2024-04-26T21:33:06.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.981+0000 7f3a4be00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 0x7f3a380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.981+0000 7f3a4be00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 0x7f3a380a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005e10 tx=0x7f3a3c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:06.985+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3a440bd670 con 0x7f3a54100ff0 2024-04-26T21:33:07.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:06 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3530440323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:07.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:06 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1240916849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:06 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3723099478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:06 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3530440323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:06 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1240916849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:06 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3723099478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.187+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a5402dd40 con 0x7f3a54100ff0 2024-04-26T21:33:07.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.188+0000 7f3a50c00640 1 -- 172.21.15.97:0/1313927937 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f3a440bcf70 con 0x7f3a54100ff0 2024-04-26T21:33:07.189 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.190+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 msgr2=0x7f3a380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.190+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 0x7f3a380a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005e10 tx=0x7f3a3c005da0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.190+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 msgr2=0x7f3a5419ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.190+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009230 tx=0x7f3a440042f0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 shutdown_connections 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a54106310 0x7f3a541a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a54103960 0x7f3a5419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3a380a3c40 0x7f3a380a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 --2- 172.21.15.97:0/1313927937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a54100ff0 0x7f3a5419ee70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 >> 172.21.15.97:0/1313927937 conn(0x7f3a540fcbf0 msgr2=0x7f3a54104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 shutdown_connections 2024-04-26T21:33:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.191+0000 7f3a58c00640 1 -- 172.21.15.97:0/1313927937 wait complete. 2024-04-26T21:33:07.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:33:07.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:06 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3530440323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:07.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:06 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1240916849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:06 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3723099478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 -- 172.21.15.97:0/969995759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff974108630 msgr2=0x7ff97410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 --2- 172.21.15.97:0/969995759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff974108630 0x7ff97410aa80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009ca0 tx=0x7ff96c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 -- 172.21.15.97:0/969995759 shutdown_connections 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 --2- 172.21.15.97:0/969995759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff974108630 0x7ff97410aa80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 --2- 172.21.15.97:0/969995759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 0x7ff9741080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 --2- 172.21.15.97:0/969995759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9741033d0 0x7ff9741057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 -- 172.21.15.97:0/969995759 >> 172.21.15.97:0/969995759 conn(0x7ff9740fcda0 msgr2=0x7ff9740ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 -- 172.21.15.97:0/969995759 shutdown_connections 2024-04-26T21:33:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.407+0000 7ff97c200640 1 -- 172.21.15.97:0/969995759 wait complete. 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 Processor -- start 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 -- start start 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 0x7ff97419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff974108630 0x7ff9741a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e380 con 0x7ff9741033d0 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e200 con 0x7ff974105d00 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff97c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e500 con 0x7ff974108630 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff979800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff979800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44644/0 (socket says 172.21.15.97:44644) 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.408+0000 7ff979800640 1 -- 172.21.15.97:0/22294794 learned_addr learned my addr 172.21.15.97:0/22294794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff978e00640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 0x7ff97419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff97a200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff974108630 0x7ff9741a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 -- 172.21.15.97:0/22294794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff974108630 msgr2=0x7ff9741a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff974108630 0x7ff9741a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 -- 172.21.15.97:0/22294794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 msgr2=0x7ff97419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 0x7ff97419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 -- 172.21.15.97:0/22294794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9741a27f0 con 0x7ff9741033d0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff979800640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c920 tx=0x7ff96800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007c70 con 0x7ff9741033d0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff968007e10 con 0x7ff9741033d0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968018710 con 0x7ff9741033d0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.409+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9741a2ae0 con 0x7ff9741033d0 2024-04-26T21:33:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.410+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a3080 con 0x7ff9741033d0 2024-04-26T21:33:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.411+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97410a120 con 0x7ff9741033d0 2024-04-26T21:33:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.413+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff96800f450 con 0x7ff9741033d0 2024-04-26T21:33:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.415+0000 7ff972000640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 0x7ff95c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.415+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff9680d0750 con 0x7ff9741033d0 2024-04-26T21:33:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.415+0000 7ff978e00640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 0x7ff95c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.416+0000 7ff978e00640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 0x7ff95c0a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff960005fd0 tx=0x7ff960005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.419+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff968014030 con 0x7ff9741033d0 2024-04-26T21:33:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.659+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff97402dad0 con 0x7ff9741033d0 2024-04-26T21:33:07.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.661+0000 7ff972000640 1 -- 172.21.15.97:0/22294794 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11937 (secure 0 0 0) 0x7ff97402dad0 con 0x7ff9741033d0 2024-04-26T21:33:07.662 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:33:07.662 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:31:00.694971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34400,24487,34433,34502,34460]},"id":1}]} 2024-04-26T21:33:07.663 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-26T21:33:07.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 msgr2=0x7ff95c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 0x7ff95c0a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff960005fd0 tx=0x7ff960005850 comp rx=0 tx=0).stop 2024-04-26T21:33:07.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 msgr2=0x7ff97419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c920 tx=0x7ff96800cde0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 shutdown_connections 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff974108630 0x7ff9741a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff974105d00 0x7ff97419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff95c0a3cc0 0x7ff95c0a6180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 --2- 172.21.15.97:0/22294794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9741033d0 0x7ff97419abd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 >> 172.21.15.97:0/22294794 conn(0x7ff9740fcda0 msgr2=0x7ff974102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 shutdown_connections 2024-04-26T21:33:07.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.664+0000 7ff97c200640 1 -- 172.21.15.97:0/22294794 wait complete. 2024-04-26T21:33:07.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:33:07.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 -- 172.21.15.97:0/981879973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c108d90 msgr2=0x7f5c2c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 --2- 172.21.15.97:0/981879973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c108d90 0x7f5c2c1091b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009900 tx=0x7f5c1c02a410 comp rx=0 tx=0).stop 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 -- 172.21.15.97:0/981879973 shutdown_connections 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 --2- 172.21.15.97:0/981879973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c2c10a940 0x7f5c2c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 --2- 172.21.15.97:0/981879973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c109f80 0x7f5c2c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 --2- 172.21.15.97:0/981879973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c108d90 0x7f5c2c1091b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 -- 172.21.15.97:0/981879973 >> 172.21.15.97:0/981879973 conn(0x7f5c2c077d30 msgr2=0x7f5c2c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 -- 172.21.15.97:0/981879973 shutdown_connections 2024-04-26T21:33:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.868+0000 7f5c30a00640 1 -- 172.21.15.97:0/981879973 wait complete. 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.869+0000 7f5c30a00640 1 Processor -- start 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.869+0000 7f5c30a00640 1 -- start start 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 0x7f5c2c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c2c10a940 0x7f5c2c1a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c114790 con 0x7f5c2c109f80 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c114610 con 0x7f5c2c108d90 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c30a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c114910 con 0x7f5c2c10a940 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44654/0 (socket says 172.21.15.97:44654) 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 -- 172.21.15.97:0/1716832227 learned_addr learned my addr 172.21.15.97:0/1716832227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c2a800640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 0x7f5c2c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 -- 172.21.15.97:0/1716832227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c2c10a940 msgr2=0x7f5c2c1a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c2c10a940 0x7f5c2c1a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 -- 172.21.15.97:0/1716832227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 msgr2=0x7f5c2c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 0x7f5c2c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c23e00640 1 -- 172.21.15.97:0/1716832227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c1a6ae0 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.870+0000 7f5c2a800640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 0x7f5c2c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c23e00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c890 tx=0x7f5c1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14004220 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c14007e40 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14004a00 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c1a6dd0 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.871+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a7240 con 0x7f5c2c109f80 2024-04-26T21:33:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.872+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c110270 con 0x7f5c2c109f80 2024-04-26T21:33:07.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.874+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5c140079b0 con 0x7f5c2c109f80 2024-04-26T21:33:07.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.875+0000 7f5c23400640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 0x7f5c0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:07.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.876+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5c140d0280 con 0x7f5c2c109f80 2024-04-26T21:33:07.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.876+0000 7f5c2a800640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 0x7f5c0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:07.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.876+0000 7f5c2a800640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 0x7f5c0c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c02a930 tx=0x7f5c1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:07.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:07.879+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5c14014030 con 0x7f5c2c109f80 2024-04-26T21:33:08.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:07 smithi097 ceph-mon[28127]: pgmap v433: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:33:08.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1313927937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:08.066 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:07 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/22294794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:08.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.066+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c2c02dd30 con 0x7f5c2c109f80 2024-04-26T21:33:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.069+0000 7f5c23400640 1 -- 172.21.15.97:0/1716832227 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f5c1409cb10 con 0x7f5c2c109f80 2024-04-26T21:33:08.071 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:33:08.071 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:33:08.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 msgr2=0x7f5c0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 0x7f5c0c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c02a930 tx=0x7f5c1c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 msgr2=0x7f5c2c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c890 tx=0x7f5c1400cd50 comp rx=0 tx=0).stop 2024-04-26T21:33:08.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 shutdown_connections 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c2c10a940 0x7f5c2c1a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5c0c0a3cc0 0x7f5c0c0a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c109f80 0x7f5c2c19f450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 --2- 172.21.15.97:0/1716832227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c2c108d90 0x7f5c2c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.071+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 >> 172.21.15.97:0/1716832227 conn(0x7f5c2c077d30 msgr2=0x7f5c2c10d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.072+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 shutdown_connections 2024-04-26T21:33:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.072+0000 7f5c30a00640 1 -- 172.21.15.97:0/1716832227 wait complete. 2024-04-26T21:33:08.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:30:53.245300+0000', 'modified': '2024-04-26T21:31:00.694971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34502, 'mds_1': 34460, 'mds_2': 34433, 'mds_3': 34400, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/3674554302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3674554302}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3674554302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/1900627986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 1900627986}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 1900627986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6833/845773802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 845773802}, {'type': 'v1', 'addr': '172.21.15.143:6833', 'nonce': 845773802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/4086222297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 4086222297}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 4086222297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/1658655529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 1658655529}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 1658655529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.143:6839/2048200369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2048200369}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2048200369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34400': {'gid': 34400, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.111:6833/4057011733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 4057011733}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 4057011733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34433': {'gid': 34433, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6835/738887742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 738887742}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 738887742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.111:6837/1944730061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1944730061}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1944730061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34502': {'gid': 34502, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2198422773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2198422773}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2198422773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34502, 'qdb_cluster': [34400, 24487, 34433, 34502, 34460]} 2024-04-26T21:33:08.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:33:08.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:33:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:07 smithi111 ceph-mon[31001]: pgmap v433: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:33:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1313927937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:07 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/22294794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 -- 172.21.15.97:0/3138703866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78106540 msgr2=0x7f6b7810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3138703866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78106540 0x7f6b7810cdd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009ca0 tx=0x7f6b7002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 -- 172.21.15.97:0/3138703866 shutdown_connections 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3138703866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78106540 0x7f6b7810cdd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3138703866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78105b80 0x7f6b78106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3138703866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78104990 0x7f6b78104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 -- 172.21.15.97:0/3138703866 >> 172.21.15.97:0/3138703866 conn(0x7f6b780fff00 msgr2=0x7f6b78102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 -- 172.21.15.97:0/3138703866 shutdown_connections 2024-04-26T21:33:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.282+0000 7f6b77e00640 1 -- 172.21.15.97:0/3138703866 wait complete. 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.283+0000 7f6b77e00640 1 Processor -- start 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.283+0000 7f6b77e00640 1 -- start start 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78104990 0x7f6b7819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78106540 0x7f6b781a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b781103a0 con 0x7f6b78105b80 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78110220 con 0x7f6b78106540 2024-04-26T21:33:08.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78110520 con 0x7f6b78104990 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44672/0 (socket says 172.21.15.97:44672) 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3981523261 learned_addr learned my addr 172.21.15.97:0/3981523261 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b77400640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78106540 0x7f6b781a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78104990 0x7f6b7819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3981523261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78104990 msgr2=0x7f6b7819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78104990 0x7f6b7819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3981523261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78106540 msgr2=0x7f6b781a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78106540 0x7f6b781a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.284+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3981523261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b781a25d0 con 0x7f6b78105b80 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c920 tx=0x7f6b6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b6f400640 1 -- 172.21.15.97:0/3981523261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60007c70 con 0x7f6b78105b80 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b6f400640 1 -- 172.21.15.97:0/3981523261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b60007e10 con 0x7f6b78105b80 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b781a28c0 con 0x7f6b78105b80 2024-04-26T21:33:08.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b6f400640 1 -- 172.21.15.97:0/3981523261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60018710 con 0x7f6b78105b80 2024-04-26T21:33:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.285+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b781a2e60 con 0x7f6b78105b80 2024-04-26T21:33:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.288+0000 7f6b809fb740 1 Processor -- start 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.288+0000 7f6b809fb740 1 -- start start 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.288+0000 7f6b6f400640 1 -- 172.21.15.97:0/3981523261 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b6000f450 con 0x7f6b78105b80 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x56474395e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x5647439637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5647438c2600 con 0x564743963e20 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56474385f100 con 0x56474395ecb0 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b809fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5647438be5c0 con 0x56474395e2c0 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44678/0 (socket says 172.21.15.97:44678) 2024-04-26T21:33:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 -- 172.21.15.97:0/1148152439 learned_addr learned my addr 172.21.15.97:0/1148152439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b76a00640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x5647439637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b77400640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x56474395e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 -- 172.21.15.97:0/1148152439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 msgr2=0x56474395e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x56474395e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 -- 172.21.15.97:0/1148152439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 msgr2=0x5647439637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x5647439637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b6fe00640 1 -- 172.21.15.97:0/1148152439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647435d42d0 con 0x564743963e20 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.289+0000 7f6b76a00640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x5647439637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b600ce060 tx=0x7f6b600d6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3ce560b330d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b6f400640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 0x7f6b5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b67e00640 1 -- 172.21.15.97:0/1148152439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b600df070 con 0x564743963e20 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b67e00640 1 -- 172.21.15.97:0/1148152439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b6009cff0 con 0x564743963e20 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b67e00640 1 -- 172.21.15.97:0/1148152439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6009c250 con 0x564743963e20 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b6f400640 1 -- 172.21.15.97:0/3981523261 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f6b600d06e0 con 0x7f6b78105b80 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.290+0000 7f6b77400640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 0x7f6b5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 -- 172.21.15.97:0/1148152439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 msgr2=0x5647439662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b600ce060 tx=0x7f6b600d6cb0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b77400640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 0x7f6b5c0a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56474395f510 tx=0x7f6b70031980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 -- 172.21.15.97:0/1148152439 shutdown_connections 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x5647439662b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x5647439637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 --2- 172.21.15.97:0/1148152439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x56474395e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 -- 172.21.15.97:0/1148152439 >> 172.21.15.97:0/1148152439 conn(0x5647438f2d10 msgr2=0x564743959d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 -- 172.21.15.97:0/1148152439 shutdown_connections 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.291+0000 7f6b809fb740 1 -- 172.21.15.97:0/1148152439 wait complete. 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 -- start start 2024-04-26T21:33:08.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 Processor -- start 2024-04-26T21:33:08.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:07 smithi143 ceph-mon[31461]: pgmap v433: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:33:08.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1313927937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:33:08.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:07 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/22294794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x5647439686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x564743968c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564743966dd0 con 0x564743963e20 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564743849f70 con 0x56474395ecb0 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564743966f50 con 0x56474395e2c0 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b809fb740 10 client.0 _ll_register_callbacks cb 0x5647438b8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b77400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b77400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44680/0 (socket says 172.21.15.97:44680) 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.292+0000 7f6b77400640 1 -- 172.21.15.97:0/3126247495 learned_addr learned my addr 172.21.15.97:0/3126247495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x5647439686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x564743968c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 -- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 msgr2=0x5647439686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x5647439686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 -- 172.21.15.97:0/3126247495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 msgr2=0x564743968c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x564743968c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 -- 172.21.15.97:0/3126247495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474369e6b0 con 0x564743963e20 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b77400640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b70003bf0 tx=0x7f6b700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1b4af9e141f9c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.293+0000 7f6b64c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7004e070 con 0x564743963e20 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f6b700305c0 con 0x564743963e20 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b6009cff0 con 0x564743963e20 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b7005a050 con 0x564743963e20 2024-04-26T21:33:08.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b809fb740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564743972bc0 con 0x564743963e20 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b6009cff0 con 0x564743963e20 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6b70043040 con 0x564743963e20 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.294+0000 7f6b809fb740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b809fb740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b809fb740 20 client.15279 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439753c0 0x564743976830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b809fb740 4 client.15279 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 --> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647438d0210 con 0x5647439753c0 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.295+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439753c0 0x564743976830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.296+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439753c0 0x564743976830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8182ddab6744e4d1 server_cookie=157b7fae68098ba1 in_seq=0 out_seq=0 2024-04-26T21:33:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.296+0000 7f6b64c00640 10 client.15279 ms_handle_connect on v2:172.21.15.143:6838/2048200369 2024-04-26T21:33:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.298+0000 7f6b64c00640 1 -- 172.21.15.97:0/3126247495 <== mds.4 v2:172.21.15.143:6838/2048200369 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647438d0210 con 0x5647439753c0 2024-04-26T21:33:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.298+0000 7f6b64c00640 10 client.15279 handle_command_reply: tid=0 2024-04-26T21:33:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.298+0000 7f6b809fb740 1 client.15279 shutdown 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 msgr2=0x56474396fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b70003bf0 tx=0x7f6b700095c0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 shutdown_connections 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439753c0 0x564743976830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396fc70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x564743968c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3126247495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474395e2c0 0x5647439686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b64c00640 0 client.15279 ms_handle_reset on v2:172.21.15.143:6838/2048200369 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 >> 172.21.15.97:0/3126247495 conn(0x5647438f2d10 msgr2=0x564743958f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 shutdown_connections 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 1 -- 172.21.15.97:0/3126247495 wait complete. 2024-04-26T21:33:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.299+0000 7f6b809fb740 20 client.15279 trim_cache size 0 max 16384 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.317+0000 7f6b809fb740 1 Processor -- start 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 -- start start 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564743963e20 0x564743973f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564743974b70 con 0x56474395e2c0 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564743975240 con 0x56474395ecb0 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b809fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564743975910 con 0x564743963e20 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44688/0 (socket says 172.21.15.97:44688) 2024-04-26T21:33:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 -- 172.21.15.97:0/2978153303 learned_addr learned my addr 172.21.15.97:0/2978153303 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 -- 172.21.15.97:0/2978153303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564743963e20 msgr2=0x564743973f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564743963e20 0x564743973f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 -- 172.21.15.97:0/2978153303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 msgr2=0x56474396fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.318+0000 7f6b76a00640 1 -- 172.21.15.97:0/2978153303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474395f540 con 0x56474395e2c0 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.319+0000 7f6b76a00640 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800ca00 tx=0x7f6b6800cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8376594d8feacb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.319+0000 7f6b67400640 1 -- 172.21.15.97:0/2978153303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68010ce0 con 0x56474395e2c0 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.319+0000 7f6b67400640 1 -- 172.21.15.97:0/2978153303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b68020030 con 0x56474395e2c0 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.319+0000 7f6b67400640 1 -- 172.21.15.97:0/2978153303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68003870 con 0x56474395e2c0 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- 172.21.15.97:0/2978153303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 msgr2=0x564743954df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800ca00 tx=0x7f6b6800cfa0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- 172.21.15.97:0/2978153303 shutdown_connections 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564743963e20 0x564743973f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 --2- 172.21.15.97:0/2978153303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56474395e2c0 0x564743954df0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- 172.21.15.97:0/2978153303 >> 172.21.15.97:0/2978153303 conn(0x5647438f2d10 msgr2=0x564743960070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- 172.21.15.97:0/2978153303 shutdown_connections 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- 172.21.15.97:0/2978153303 wait complete. 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 -- start start 2024-04-26T21:33:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.320+0000 7f6b809fb740 1 Processor -- start 2024-04-26T21:33:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474396efc0 0x56474396f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5647439766b0 con 0x564743963e20 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564743975fe0 con 0x56474395ecb0 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564743976d80 con 0x56474396efc0 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b809fb740 10 client.0 _ll_register_callbacks cb 0x7f6b70009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44702/0 (socket says 172.21.15.97:44702) 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3180515422 learned_addr learned my addr 172.21.15.97:0/3180515422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474396efc0 msgr2=0x56474396f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474396efc0 0x56474396f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3180515422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 msgr2=0x56474396ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.321+0000 7f6b6fe00640 1 -- 172.21.15.97:0/3180515422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474396f980 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b6fe00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b6009c830 tx=0x7f6b600ce090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaa5d89e11f6738 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6006c800 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f6b6000fa70 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b68020030 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b6006c9e0 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b809fb740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647439dbd90 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b68020030 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6b60098f60 con 0x564743963e20 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.322+0000 7f6b809fb740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b809fb740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:33:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b809fb740 20 client.15285 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439e04f0 0x5647439dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b809fb740 4 client.15285 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 --> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] -- command(tid 0: {"prefix": "status"}) -- 0x5647438ca080 con 0x5647439e04f0 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.323+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439e04f0 0x5647439dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.324+0000 7f6b76a00640 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439e04f0 0x5647439dc930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f90895b7613e5c1 server_cookie=a5cd42a7f673f3d3 in_seq=0 out_seq=0 2024-04-26T21:33:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.324+0000 7f6b64c00640 10 client.15285 ms_handle_connect on v2:172.21.15.143:6838/2048200369 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b64c00640 1 -- 172.21.15.97:0/3180515422 <== mds.4 v2:172.21.15.143:6838/2048200369 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5647438ca080 con 0x5647439e04f0 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b64c00640 10 client.15285 handle_command_reply: tid=0 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 client.15285 shutdown 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 msgr2=0x56474396bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b6009c830 tx=0x7f6b600ce090 comp rx=0 tx=0).stop 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 shutdown_connections 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:6838/2048200369,v1:172.21.15.143:6839/2048200369] conn(0x5647439e04f0 0x5647439dc930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56474396efc0 0x56474396f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564743963e20 0x56474396bfa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 --2- 172.21.15.97:0/3180515422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56474395ecb0 0x56474396ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b64c00640 0 client.15285 ms_handle_reset on v2:172.21.15.143:6838/2048200369 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 >> 172.21.15.97:0/3180515422 conn(0x5647438f2d10 msgr2=0x564743973f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 shutdown_connections 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 1 -- 172.21.15.97:0/3180515422 wait complete. 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.325+0000 7f6b809fb740 20 client.15285 trim_cache size 0 max 16384 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24487, 2024-04-26T21:33:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 128.00303925599999, 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 152.988074127 2024-04-26T21:33:08.330 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 msgr2=0x7f6b5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 0x7f6b5c0a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56474395f510 tx=0x7f6b70031980 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 msgr2=0x7f6b7819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c920 tx=0x7f6b6000cde0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 shutdown_connections 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b78106540 0x7f6b781a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f6b5c0a3cc0 0x7f6b5c0a6180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b78105b80 0x7f6b7819aef0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 --2- 172.21.15.97:0/3981523261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b78104990 0x7f6b7819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 >> 172.21.15.97:0/3981523261 conn(0x7f6b780fff00 msgr2=0x7f6b78108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 shutdown_connections 2024-04-26T21:33:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.326+0000 7f6b77e00640 1 -- 172.21.15.97:0/3981523261 wait complete. 2024-04-26T21:33:08.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:33:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 -- 172.21.15.97:0/1005237886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 msgr2=0x7ff89c1113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 --2- 172.21.15.97:0/1005237886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1113f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff894009ca0 tx=0x7ff89402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 -- 172.21.15.97:0/1005237886 shutdown_connections 2024-04-26T21:33:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 --2- 172.21.15.97:0/1005237886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1113f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 --2- 172.21.15.97:0/1005237886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 0x7ff89c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 --2- 172.21.15.97:0/1005237886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 0x7ff89c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 -- 172.21.15.97:0/1005237886 >> 172.21.15.97:0/1005237886 conn(0x7ff89c077f40 msgr2=0x7ff89c07a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 -- 172.21.15.97:0/1005237886 shutdown_connections 2024-04-26T21:33:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.543+0000 7ff8a1600640 1 -- 172.21.15.97:0/1005237886 wait complete. 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 Processor -- start 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 -- start start 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 0x7ff89c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 0x7ff89c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1149b0 con 0x7ff89c10ab60 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c114830 con 0x7ff89c108fb0 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.544+0000 7ff8a1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c114b30 con 0x7ff89c10a1a0 2024-04-26T21:33:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44728/0 (socket says 172.21.15.97:44728) 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 -- 172.21.15.97:0/2004109444 learned_addr learned my addr 172.21.15.97:0/2004109444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff893e00640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 0x7ff89c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89aa00640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 0x7ff89c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 -- 172.21.15.97:0/2004109444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 msgr2=0x7ff89c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 0x7ff89c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 -- 172.21.15.97:0/2004109444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 msgr2=0x7ff89c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 0x7ff89c19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 -- 172.21.15.97:0/2004109444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1a6cb0 con 0x7ff89c10ab60 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff89b400640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff894031040 tx=0x7ff894003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.545+0000 7ff893400640 1 -- 172.21.15.97:0/2004109444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894004040 con 0x7ff89c10ab60 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.546+0000 7ff893400640 1 -- 172.21.15.97:0/2004109444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff894004670 con 0x7ff89c10ab60 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.546+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c1a6f40 con 0x7ff89c10ab60 2024-04-26T21:33:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.546+0000 7ff893400640 1 -- 172.21.15.97:0/2004109444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894030d60 con 0x7ff89c10ab60 2024-04-26T21:33:08.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.546+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c1a7400 con 0x7ff89c10ab60 2024-04-26T21:33:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.549+0000 7ff893400640 1 -- 172.21.15.97:0/2004109444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff8940305c0 con 0x7ff89c10ab60 2024-04-26T21:33:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.549+0000 7ff8a45de740 1 Processor -- start 2024-04-26T21:33:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.549+0000 7ff8a45de740 1 -- start start 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238faad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238fab50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56238fa11600 con 0x56238faadae0 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56238f9ae100 con 0x56238faad0f0 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff8a45de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fa0d5c0 con 0x56238fab2c50 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44734/0 (socket says 172.21.15.97:44734) 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 -- 172.21.15.97:0/3051648920 learned_addr learned my addr 172.21.15.97:0/3051648920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff893e00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238faad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff893e00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238fab50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 -- 172.21.15.97:0/3051648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 msgr2=0x56238fab50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238fab50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 -- 172.21.15.97:0/3051648920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 msgr2=0x56238faad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238faad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff89aa00640 1 -- 172.21.15.97:0/3051648920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238f7232d0 con 0x56238faadae0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.550+0000 7ff893e00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238faad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009b70 tx=0x7ff88c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4233b021985b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff88b400640 1 -- 172.21.15.97:0/3051648920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c016070 con 0x56238faadae0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff893400640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 0x7ff87c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff88b400640 1 -- 172.21.15.97:0/3051648920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0054b0 con 0x56238faadae0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff88b400640 1 -- 172.21.15.97:0/3051648920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c004e10 con 0x56238faadae0 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff893400640 1 -- 172.21.15.97:0/2004109444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff8940f1a80 con 0x7ff89c10ab60 2024-04-26T21:33:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.551+0000 7ff893e00640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 0x7ff87c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 -- 172.21.15.97:0/3051648920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 msgr2=0x56238fab25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009b70 tx=0x7ff88c00e6c0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff893e00640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 0x7ff87c0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56238faae340 tx=0x7ff884004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 -- 172.21.15.97:0/3051648920 shutdown_connections 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238fab50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faadae0 0x56238fab25e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3051648920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238faad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 -- 172.21.15.97:0/3051648920 >> 172.21.15.97:0/3051648920 conn(0x56238fa41d10 msgr2=0x56238faa8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 -- 172.21.15.97:0/3051648920 shutdown_connections 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.552+0000 7ff8a45de740 1 -- 172.21.15.97:0/3051648920 wait complete. 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 -- start start 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 Processor -- start 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238fab7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fab7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56238fab5c00 con 0x56238fab2c50 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56238f998f70 con 0x56238faad0f0 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fab5d80 con 0x56238faadae0 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff8a45de740 10 client.0 _ll_register_callbacks cb 0x56238fa07480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff89b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff89b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44748/0 (socket says 172.21.15.97:44748) 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.553+0000 7ff89b400640 1 -- 172.21.15.97:0/4104946203 learned_addr learned my addr 172.21.15.97:0/4104946203 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff893e00640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fab7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89aa00640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238fab7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 -- 172.21.15.97:0/4104946203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 msgr2=0x56238fab7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fab7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 -- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 msgr2=0x56238fab7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238fab7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 -- 172.21.15.97:0/4104946203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238f7ed6b0 con 0x56238fab2c50 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff89b400640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff894045000 tx=0x7ff8940bd740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a5744220658a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff883e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8940c20d0 con 0x56238fab2c50 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff8940ba390 con 0x56238fab2c50 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff88c0054b0 con 0x56238fab2c50 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.554+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8940c6990 con 0x56238fab2c50 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56238fab5c00 con 0x56238fab2c50 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0054b0 con 0x56238fab2c50 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff8940c6500 con 0x56238fab2c50 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 10 client.15297 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 20 client.15297 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fac4160 0x56238fac55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 4 client.15297 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.555+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 --> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56238fa1f210 con 0x56238fac4160 2024-04-26T21:33:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.556+0000 7ff89aa00640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fac4160 0x56238fac55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.557+0000 7ff89aa00640 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fac4160 0x56238fac55d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b464689c869118f7 server_cookie=80cfba2f88546fe7 in_seq=0 out_seq=0 2024-04-26T21:33:08.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.557+0000 7ff883e00640 10 client.15297 ms_handle_connect on v2:172.21.15.111:6832/4057011733 2024-04-26T21:33:08.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff883e00640 1 -- 172.21.15.97:0/4104946203 <== mds.3 v2:172.21.15.111:6832/4057011733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56238fa1f210 con 0x56238fac4160 2024-04-26T21:33:08.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff883e00640 10 client.15297 handle_command_reply: tid=0 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 client.15297 shutdown 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 msgr2=0x56238fabea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff894045000 tx=0x7ff8940bd740 comp rx=0 tx=0).stop 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 shutdown_connections 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fac4160 0x56238fac55d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238fab2c50 0x56238fabea90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fab7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 --2- 172.21.15.97:0/4104946203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faad0f0 0x56238fab7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff883e00640 0 client.15297 ms_handle_reset on v2:172.21.15.111:6832/4057011733 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 >> 172.21.15.97:0/4104946203 conn(0x56238fa41d10 msgr2=0x56238faa7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 shutdown_connections 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 1 -- 172.21.15.97:0/4104946203 wait complete. 2024-04-26T21:33:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.559+0000 7ff8a45de740 20 client.15297 trim_cache size 0 max 16384 2024-04-26T21:33:08.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 Processor -- start 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 -- start start 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faadae0 0x56238faa7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238faa7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56238fac3910 con 0x56238faad0f0 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56238fac3fe0 con 0x56238faadae0 2024-04-26T21:33:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff8a45de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fac46b0 con 0x56238fab2c50 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44762/0 (socket says 172.21.15.97:44762) 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.579+0000 7ff89aa00640 1 -- 172.21.15.97:0/95084802 learned_addr learned my addr 172.21.15.97:0/95084802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 -- 172.21.15.97:0/95084802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 msgr2=0x56238faa7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238faa7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 -- 172.21.15.97:0/95084802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faadae0 msgr2=0x56238faa7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faadae0 0x56238faa7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 -- 172.21.15.97:0/95084802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238faa9820 con 0x56238faad0f0 2024-04-26T21:33:08.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff89aa00640 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009a90 tx=0x7ff88c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cd0a4a978c87e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff88aa00640 1 -- 172.21.15.97:0/95084802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c016040 con 0x56238faad0f0 2024-04-26T21:33:08.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff88aa00640 1 -- 172.21.15.97:0/95084802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0274b0 con 0x56238faad0f0 2024-04-26T21:33:08.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.580+0000 7ff88aa00640 1 -- 172.21.15.97:0/95084802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0118b0 con 0x56238faad0f0 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- 172.21.15.97:0/95084802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 msgr2=0x56238fac3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009a90 tx=0x7ff88c005900 comp rx=0 tx=0).stop 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- 172.21.15.97:0/95084802 shutdown_connections 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fab2c50 0x56238faa7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238faadae0 0x56238faa7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 --2- 172.21.15.97:0/95084802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fac3270 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- 172.21.15.97:0/95084802 >> 172.21.15.97:0/95084802 conn(0x56238fa41d10 msgr2=0x56238fac2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- 172.21.15.97:0/95084802 shutdown_connections 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- 172.21.15.97:0/95084802 wait complete. 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 -- start start 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.581+0000 7ff8a45de740 1 Processor -- start 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fabe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238fab2c50 0x56238fabac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56238fac5450 con 0x56238faad0f0 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56238fac4d80 con 0x56238fab2c50 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fac5b20 con 0x56238faadae0 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff8a45de740 10 client.0 _ll_register_callbacks cb 0x7ff894009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44778/0 (socket says 172.21.15.97:44778) 2024-04-26T21:33:08.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 -- 172.21.15.97:0/3914179328 learned_addr learned my addr 172.21.15.97:0/3914179328 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 -- 172.21.15.97:0/3914179328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 msgr2=0x56238fabe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fabe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 -- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238fab2c50 msgr2=0x56238fabac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238fab2c50 0x56238fabac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.582+0000 7ff89aa00640 1 -- 172.21.15.97:0/3914179328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238fabb3f0 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff89aa00640 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0059f0 tx=0x7ff88c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12183910203402d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff883e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c016040 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7ff88c027020 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff88c0274b0 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff88c027260 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff8a45de740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56238fb2ade0 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.583+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0274b0 con 0x56238faad0f0 2024-04-26T21:33:08.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff88c00f040 con 0x56238faad0f0 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 10 client.15303 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 20 client.15303 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fb2f3e0 0x56238fb2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 4 client.15303 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 --> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] -- command(tid 0: {"prefix": "status"}) -- 0x56238fa19080 con 0x56238fb2f3e0 2024-04-26T21:33:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.584+0000 7ff893e00640 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fb2f3e0 0x56238fb2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.585+0000 7ff893e00640 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fb2f3e0 0x56238fb2e0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c5127e0eeddd77f9 server_cookie=5bfa65ffb894e0f2 in_seq=0 out_seq=0 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.585+0000 7ff883e00640 10 client.15303 ms_handle_connect on v2:172.21.15.111:6832/4057011733 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff883e00640 1 -- 172.21.15.97:0/3914179328 <== mds.3 v2:172.21.15.111:6832/4057011733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56238fa19080 con 0x56238fb2f3e0 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff883e00640 10 client.15303 handle_command_reply: tid=0 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 client.15303 shutdown 2024-04-26T21:33:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 msgr2=0x56238fabe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0059f0 tx=0x7ff88c009e40 comp rx=0 tx=0).stop 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 shutdown_connections 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:6832/4057011733,v1:172.21.15.111:6833/4057011733] conn(0x56238fb2f3e0 0x56238fb2e0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56238fab2c50 0x56238fabac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238faadae0 0x56238fabe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 --2- 172.21.15.97:0/3914179328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56238faad0f0 0x56238fabe1c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff883e00640 0 client.15303 ms_handle_reset on v2:172.21.15.111:6832/4057011733 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.586+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 >> 172.21.15.97:0/3914179328 conn(0x56238fa41d10 msgr2=0x56238faaba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 shutdown_connections 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a45de740 1 -- 172.21.15.97:0/3914179328 wait complete. 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a45de740 20 client.15303 trim_cache size 0 max 16384 2024-04-26T21:33:08.590 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34400, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 129.27841559500001, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 181.569307635 2024-04-26T21:33:08.591 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 msgr2=0x7ff87c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 0x7ff87c0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56238faae340 tx=0x7ff884004730 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 msgr2=0x7ff89c1a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff894031040 tx=0x7ff894003ba0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 shutdown_connections 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7ff87c0a3c40 0x7ff87c0a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89c10ab60 0x7ff89c1a6640 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff89c10a1a0 0x7ff89c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 --2- 172.21.15.97:0/2004109444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff89c108fb0 0x7ff89c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.587+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 >> 172.21.15.97:0/2004109444 conn(0x7ff89c077f40 msgr2=0x7ff89c10d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.588+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 shutdown_connections 2024-04-26T21:33:08.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.588+0000 7ff8a1600640 1 -- 172.21.15.97:0/2004109444 wait complete. 2024-04-26T21:33:08.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.788+0000 7f7edfe00640 1 -- 172.21.15.97:0/1888864575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8106760 msgr2=0x7f7ed810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.788+0000 7f7edfe00640 1 --2- 172.21.15.97:0/1888864575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8106760 0x7f7ed810cff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 -- 172.21.15.97:0/1888864575 shutdown_connections 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 --2- 172.21.15.97:0/1888864575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8106760 0x7f7ed810cff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 --2- 172.21.15.97:0/1888864575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8105da0 0x7f7ed8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 --2- 172.21.15.97:0/1888864575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 0x7f7ed8104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 -- 172.21.15.97:0/1888864575 >> 172.21.15.97:0/1888864575 conn(0x7f7ed8100120 msgr2=0x7f7ed8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 -- 172.21.15.97:0/1888864575 shutdown_connections 2024-04-26T21:33:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.789+0000 7f7edfe00640 1 -- 172.21.15.97:0/1888864575 wait complete. 2024-04-26T21:33:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 Processor -- start 2024-04-26T21:33:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 -- start start 2024-04-26T21:33:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 0x7f7ed819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8106760 0x7f7ed81a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81105e0 con 0x7f7ed8105da0 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8110460 con 0x7f7ed8106760 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edfe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8110760 con 0x7f7ed8104bb0 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44796/0 (socket says 172.21.15.97:44796) 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.790+0000 7f7edca00640 1 -- 172.21.15.97:0/567899602 learned_addr learned my addr 172.21.15.97:0/567899602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edd400640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 0x7f7ed819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 -- 172.21.15.97:0/567899602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 msgr2=0x7f7ed819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 0x7f7ed819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edde00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8106760 0x7f7ed81a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 -- 172.21.15.97:0/567899602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8106760 msgr2=0x7f7ed81a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8106760 0x7f7ed81a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 -- 172.21.15.97:0/567899602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed81a2900 con 0x7f7ed8105da0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edca00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400d950 tx=0x7f7ec400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7ed6000640 1 -- 172.21.15.97:0/567899602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4017070 con 0x7f7ed8105da0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7ed6000640 1 -- 172.21.15.97:0/567899602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ec4009c30 con 0x7f7ed8105da0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed81a2bf0 con 0x7f7ed8105da0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7ed6000640 1 -- 172.21.15.97:0/567899602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400f620 con 0x7f7ed8105da0 2024-04-26T21:33:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.791+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a3190 con 0x7f7ed8105da0 2024-04-26T21:33:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.794+0000 7f7ee2c21740 1 Processor -- start 2024-04-26T21:33:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ed6000640 1 -- 172.21.15.97:0/567899602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7ec4010460 con 0x7f7ed8105da0 2024-04-26T21:33:08.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 -- start start 2024-04-26T21:33:08.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf1a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612ace76600 con 0x5612acf120f0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612ace13100 con 0x5612acf17c50 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5612ace725c0 con 0x5612acf12ae0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44800/0 (socket says 172.21.15.97:44800) 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.795+0000 7f7edde00640 1 -- 172.21.15.97:0/464654286 learned_addr learned my addr 172.21.15.97:0/464654286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edd400640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 -- 172.21.15.97:0/464654286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 msgr2=0x5612acf175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edca00640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf1a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf175e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 -- 172.21.15.97:0/464654286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 msgr2=0x5612acf1a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf1a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 -- 172.21.15.97:0/464654286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612acb882d0 con 0x5612acf120f0 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edca00640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf1a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:08.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7edde00640 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005ab0 tx=0x7f7ed0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93cc6dfa559ee8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7ece600640 1 -- 172.21.15.97:0/464654286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0040070 con 0x5612acf120f0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7ece600640 1 -- 172.21.15.97:0/464654286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ed0003a80 con 0x5612acf120f0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7ece600640 1 -- 172.21.15.97:0/464654286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0035030 con 0x5612acf120f0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.796+0000 7f7ed6000640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 0x7f7ebc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ed6000640 1 -- 172.21.15.97:0/567899602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f7ec40d01c0 con 0x7f7ed8105da0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7edd400640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 0x7f7ebc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 -- 172.21.15.97:0/464654286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 msgr2=0x5612acf12510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005ab0 tx=0x7f7ed0009680 comp rx=0 tx=0).stop 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 -- 172.21.15.97:0/464654286 shutdown_connections 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf1a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/464654286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf120f0 0x5612acf12510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 -- 172.21.15.97:0/464654286 >> 172.21.15.97:0/464654286 conn(0x5612acea6d10 msgr2=0x5612acf0db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7edd400640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 0x7f7ebc0a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5612acf12ee0 tx=0x7f7ec8009320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 -- 172.21.15.97:0/464654286 shutdown_connections 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.797+0000 7f7ee2c21740 1 -- 172.21.15.97:0/464654286 wait complete. 2024-04-26T21:33:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 -- start start 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 Processor -- start 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf1c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612acf1aba0 con 0x5612acf12ae0 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612acdfdf70 con 0x5612acf17c50 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.798+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5612acf1ad20 con 0x5612acf120f0 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7ee2c21740 10 client.0 _ll_register_callbacks cb 0x5612ace6c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44808/0 (socket says 172.21.15.97:44808) 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 -- 172.21.15.97:0/2492254738 learned_addr learned my addr 172.21.15.97:0/2492254738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edd400640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf1c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 -- 172.21.15.97:0/2492254738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 msgr2=0x5612acf1c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf1c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 -- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 msgr2=0x5612acf23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 -- 172.21.15.97:0/2492254738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612acc526b0 con 0x5612acf12ae0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7edca00640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40987b0 tx=0x7f7ec4009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa659c94c081fc66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7ec2a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.799+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4020070 con 0x5612acf12ae0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f7ec409b9a0 con 0x5612acf12ae0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed0003a80 con 0x5612acf12ae0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ec40a4fc0 con 0x5612acf12ae0 2024-04-26T21:33:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612acf26a60 con 0x5612acf12ae0 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ed0003a80 con 0x5612acf12ae0 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7ec4010750 con 0x5612acf12ae0 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 10 client.15315 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 20 client.15315 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf29110 0x5612acf2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 4 client.15315 mds_command: new command op to 34433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.800+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 --> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612ace84210 con 0x5612acf29110 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.801+0000 7f7edd400640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf29110 0x5612acf2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.802+0000 7f7edd400640 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf29110 0x5612acf2a580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78a222c5d6ab2338 server_cookie=f1ec92b77d7a19d5 in_seq=0 out_seq=0 2024-04-26T21:33:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.802+0000 7f7ec2a00640 10 client.15315 ms_handle_connect on v2:172.21.15.111:6834/738887742 2024-04-26T21:33:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2492254738 <== mds.2 v2:172.21.15.111:6834/738887742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612ace84210 con 0x5612acf29110 2024-04-26T21:33:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ec2a00640 10 client.15315 handle_command_reply: tid=0 2024-04-26T21:33:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 client.15315 shutdown 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 msgr2=0x5612acf1ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40987b0 tx=0x7f7ec4009de0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 shutdown_connections 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf29110 0x5612acf2a580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf17c50 0x5612acf23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf12ae0 0x5612acf1ca60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2492254738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf1c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ec2a00640 0 client.15315 ms_handle_reset on v2:172.21.15.111:6834/738887742 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 >> 172.21.15.97:0/2492254738 conn(0x5612acea6d10 msgr2=0x5612acf0ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 shutdown_connections 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2492254738 wait complete. 2024-04-26T21:33:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.804+0000 7f7ee2c21740 20 client.15315 trim_cache size 0 max 16384 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 Processor -- start 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 -- start start 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf12ae0 0x5612acf23a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612acf288c0 con 0x5612acf17c50 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612acf28f90 con 0x5612acf12ae0 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.823+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5612acf29660 con 0x5612acf120f0 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44812/0 (socket says 172.21.15.97:44812) 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 -- 172.21.15.97:0/2119415136 learned_addr learned my addr 172.21.15.97:0/2119415136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edd400640 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edca00640 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf12ae0 0x5612acf23a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 -- 172.21.15.97:0/2119415136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 msgr2=0x5612acf13580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf13580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 -- 172.21.15.97:0/2119415136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf12ae0 msgr2=0x5612acf23a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf12ae0 0x5612acf23a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 -- 172.21.15.97:0/2119415136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612acf098c0 con 0x5612acf17c50 2024-04-26T21:33:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.824+0000 7f7edde00640 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed0004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ce76a55666ba59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ecdc00640 1 -- 172.21.15.97:0/2119415136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed004d020 con 0x5612acf17c50 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ecdc00640 1 -- 172.21.15.97:0/2119415136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ed0003d40 con 0x5612acf17c50 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ecdc00640 1 -- 172.21.15.97:0/2119415136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0040070 con 0x5612acf17c50 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2119415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 msgr2=0x5612acf1b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed0004540 comp rx=0 tx=0).stop 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2119415136 shutdown_connections 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf1b810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf12ae0 0x5612acf23a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2119415136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf120f0 0x5612acf13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2119415136 >> 172.21.15.97:0/2119415136 conn(0x5612acea6d10 msgr2=0x5612acf13ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2119415136 shutdown_connections 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.825+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2119415136 wait complete. 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 -- start start 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 Processor -- start 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf120f0 0x5612acf1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf1f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612acf2a400 con 0x5612acf17c50 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612acf29d30 con 0x5612acf120f0 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5612acf2aad0 con 0x5612acf12ae0 2024-04-26T21:33:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.826+0000 7f7ee2c21740 10 client.0 _ll_register_callbacks cb 0x7f7ec400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44828/0 (socket says 172.21.15.97:44828) 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 -- 172.21.15.97:0/2461568044 learned_addr learned my addr 172.21.15.97:0/2461568044 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edca00640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf1f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 -- 172.21.15.97:0/2461568044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 msgr2=0x5612acf1f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf1f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 -- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf120f0 msgr2=0x5612acf1f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf120f0 0x5612acf1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 -- 172.21.15.97:0/2461568044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612acf22ec0 con 0x5612acf17c50 2024-04-26T21:33:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7edde00640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7ed002a430 tx=0x7f7ed0031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3dc8b971965523 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7ec2a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed004d030 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f7ed00036b0 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.827+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed0003d40 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ed0040040 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612acf8f1f0 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ed0003d40 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed003a040 con 0x5612acf17c50 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 10 client.15321 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 20 client.15321 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf93890 0x5612acf925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 4 client.15321 mds_command: new command op to 34433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:33:08.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.828+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 --> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] -- command(tid 0: {"prefix": "status"}) -- 0x5612ace7e080 con 0x5612acf93890 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.829+0000 7f7edd400640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf93890 0x5612acf925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.829+0000 7f7edd400640 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf93890 0x5612acf925a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93d87acb036bc2e0 server_cookie=c006f3279072d561 in_seq=0 out_seq=0 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.829+0000 7f7ec2a00640 10 client.15321 ms_handle_connect on v2:172.21.15.111:6834/738887742 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ec2a00640 1 -- 172.21.15.97:0/2461568044 <== mds.2 v2:172.21.15.111:6834/738887742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5612ace7e080 con 0x5612acf93890 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ec2a00640 10 client.15321 handle_command_reply: tid=0 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 client.15321 shutdown 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 msgr2=0x5612acf228d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7ed002a430 tx=0x7f7ed0031730 comp rx=0 tx=0).stop 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 shutdown_connections 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:6834/738887742,v1:172.21.15.111:6835/738887742] conn(0x5612acf93890 0x5612acf925a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612acf17c50 0x5612acf228d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612acf12ae0 0x5612acf1f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ec2a00640 0 client.15321 ms_handle_reset on v2:172.21.15.111:6834/738887742 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 --2- 172.21.15.97:0/2461568044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612acf120f0 0x5612acf1f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.830+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 >> 172.21.15.97:0/2461568044 conn(0x5612acea6d10 msgr2=0x5612acf1b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 shutdown_connections 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7ee2c21740 1 -- 172.21.15.97:0/2461568044 wait complete. 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7ee2c21740 20 client.15321 trim_cache size 0 max 16384 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-04-26T21:33:08.835 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34433, 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 130.53964708399999, 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 169.864455596 2024-04-26T21:33:08.836 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 msgr2=0x7f7ebc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 0x7f7ebc0a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5612acf12ee0 tx=0x7f7ec8009320 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 msgr2=0x7f7ed819b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400d950 tx=0x7f7ec400de10 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 shutdown_connections 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ed8106760 0x7f7ed81a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7ebc0a3cc0 0x7f7ebc0a6180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed8105da0 0x7f7ed819b200 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 --2- 172.21.15.97:0/567899602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ed8104bb0 0x7f7ed819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 >> 172.21.15.97:0/567899602 conn(0x7f7ed8100120 msgr2=0x7f7ed8109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 shutdown_connections 2024-04-26T21:33:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:08.831+0000 7f7edfe00640 1 -- 172.21.15.97:0/567899602 wait complete. 2024-04-26T21:33:08.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:33:08.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:08 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1716832227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 -- 172.21.15.97:0/1904958896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd41031e0 msgr2=0x7f5dd41055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/1904958896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd41031e0 0x7f5dd41055d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009900 tx=0x7f5dc402a410 comp rx=0 tx=0).stop 2024-04-26T21:33:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 -- 172.21.15.97:0/1904958896 shutdown_connections 2024-04-26T21:33:09.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/1904958896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd4108440 0x7f5dd410a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/1904958896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 0x7f5dd4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/1904958896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd41031e0 0x7f5dd41055d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 -- 172.21.15.97:0/1904958896 >> 172.21.15.97:0/1904958896 conn(0x7f5dd40fcbf0 msgr2=0x7f5dd40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 -- 172.21.15.97:0/1904958896 shutdown_connections 2024-04-26T21:33:09.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.032+0000 7f5dd8c00640 1 -- 172.21.15.97:0/1904958896 wait complete. 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 Processor -- start 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 -- start start 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 0x7f5dd4102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 0x7f5dd4101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410e200 con 0x7f5dd4108440 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410e080 con 0x7f5dd4105b10 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.033+0000 7f5dd8c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd410e380 con 0x7f5dd41031e0 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44852/0 (socket says 172.21.15.97:44852) 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 -- 172.21.15.97:0/195716382 learned_addr learned my addr 172.21.15.97:0/195716382 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3400640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 0x7f5dd4102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 -- 172.21.15.97:0/195716382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 msgr2=0x7f5dd4102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 0x7f5dd4102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 -- 172.21.15.97:0/195716382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 msgr2=0x7f5dd4101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 0x7f5dd4101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 0x7f5dd4101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 -- 172.21.15.97:0/195716382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd4101f90 con 0x7f5dd4108440 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3400640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 0x7f5dd4102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.034+0000 7f5dd3e00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b970 tx=0x7f5dcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.035+0000 7f5dcbe00640 1 -- 172.21.15.97:0/195716382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc019070 con 0x7f5dd4108440 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.035+0000 7f5dcbe00640 1 -- 172.21.15.97:0/195716382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dcc014440 con 0x7f5dd4108440 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.035+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd41a2ac0 con 0x7f5dd4108440 2024-04-26T21:33:09.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.035+0000 7f5dcbe00640 1 -- 172.21.15.97:0/195716382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0075d0 con 0x7f5dd4108440 2024-04-26T21:33:09.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.035+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd41a3010 con 0x7f5dd4108440 2024-04-26T21:33:09.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 Processor -- start 2024-04-26T21:33:09.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5dcbe00640 1 -- 172.21.15.97:0/195716382 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5dcc01c070 con 0x7f5dd4108440 2024-04-26T21:33:09.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 -- start start 2024-04-26T21:33:09.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd75826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd758a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbd74e6600 con 0x55cbd7582cb0 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbd7483100 con 0x55cbd7587e20 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.038+0000 7f5ddd028740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbd74e25c0 con 0x55cbd75822c0 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44854/0 (socket says 172.21.15.97:44854) 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 -- 172.21.15.97:0/640807469 learned_addr learned my addr 172.21.15.97:0/640807469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd75826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 -- 172.21.15.97:0/640807469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 msgr2=0x55cbd75826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd75826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 -- 172.21.15.97:0/640807469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 msgr2=0x55cbd758a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd758a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 -- 172.21.15.97:0/640807469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd71f82d0 con 0x55cbd7582cb0 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.039+0000 7f5dd3400640 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4000c00 tx=0x7f5dc4004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f08de6fc28ef20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dbfe00640 1 -- 172.21.15.97:0/640807469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4040070 con 0x55cbd7582cb0 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dcbe00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 0x7f5db80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dbfe00640 1 -- 172.21.15.97:0/640807469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dc4003c80 con 0x55cbd7582cb0 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dbfe00640 1 -- 172.21.15.97:0/640807469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4035030 con 0x55cbd7582cb0 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dcbe00640 1 -- 172.21.15.97:0/195716382 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5dcc0cfa60 con 0x7f5dd4108440 2024-04-26T21:33:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 0x7f5db80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 -- 172.21.15.97:0/640807469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 msgr2=0x55cbd75877b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4000c00 tx=0x7f5dc4004390 comp rx=0 tx=0).stop 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 -- 172.21.15.97:0/640807469 shutdown_connections 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd758a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd75877b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 --2- 172.21.15.97:0/640807469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd75826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5ddd028740 1 -- 172.21.15.97:0/640807469 >> 172.21.15.97:0/640807469 conn(0x55cbd7516d10 msgr2=0x55cbd757dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.040+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 0x7f5db80a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005b80 tx=0x7f5dc0005b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.041+0000 7f5ddd028740 1 -- 172.21.15.97:0/640807469 shutdown_connections 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.041+0000 7f5ddd028740 1 -- 172.21.15.97:0/640807469 wait complete. 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.041+0000 7f5ddd028740 1 -- start start 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.041+0000 7f5ddd028740 1 Processor -- start 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd75822c0 0x55cbd758c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7587e20 0x55cbd7593c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbd758ad70 con 0x55cbd7582cb0 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbd746df70 con 0x55cbd75822c0 2024-04-26T21:33:09.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbd758aef0 con 0x55cbd7587e20 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5ddd028740 10 client.0 _ll_register_callbacks cb 0x55cbd74dc480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44866/0 (socket says 172.21.15.97:44866) 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 -- 172.21.15.97:0/1074287843 learned_addr learned my addr 172.21.15.97:0/1074287843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 -- 172.21.15.97:0/1074287843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7587e20 msgr2=0x55cbd7593c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7587e20 0x55cbd7593c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 -- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd75822c0 msgr2=0x55cbd758c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd75822c0 0x55cbd758c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 -- 172.21.15.97:0/1074287843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd72c26b0 con 0x55cbd7582cb0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00116f0 tx=0x7f5dc0005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4383bf43f294a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.042+0000 7f5dbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0022070 con 0x55cbd7582cb0 2024-04-26T21:33:09.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5dc00108a0 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc4003c80 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dc002e050 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5ddd028740 10 client.15333 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbd7596d10 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dc4003c80 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.043+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dc001eb60 con 0x55cbd7582cb0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 10 client.15333 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 20 client.15333 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7599530 0x55cbd759a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 4 client.15333 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 --> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd74f4210 con 0x55cbd7599530 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.044+0000 7f5dd3400640 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7599530 0x55cbd759a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.045+0000 7f5dd3400640 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7599530 0x55cbd759a9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22ed86d924aaacda server_cookie=5b6fed84b4e79f3 in_seq=0 out_seq=0 2024-04-26T21:33:09.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.045+0000 7f5dbcc00640 10 client.15333 ms_handle_connect on v2:172.21.15.111:6836/1944730061 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5dbcc00640 1 -- 172.21.15.97:0/1074287843 <== mds.1 v2:172.21.15.111:6836/1944730061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbd74f4210 con 0x55cbd7599530 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5dbcc00640 10 client.15333 handle_command_reply: tid=0 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 client.15333 shutdown 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 msgr2=0x55cbd758cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00116f0 tx=0x7f5dc0005aa0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 shutdown_connections 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7599530 0x55cbd759a9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7587e20 0x55cbd7593c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758cbd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5dbcc00640 0 client.15333 ms_handle_reset on v2:172.21.15.111:6836/1944730061 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 --2- 172.21.15.97:0/1074287843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd75822c0 0x55cbd758c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.047+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 >> 172.21.15.97:0/1074287843 conn(0x55cbd7516d10 msgr2=0x55cbd757cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.048+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 shutdown_connections 2024-04-26T21:33:09.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.048+0000 7f5ddd028740 1 -- 172.21.15.97:0/1074287843 wait complete. 2024-04-26T21:33:09.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.048+0000 7f5ddd028740 20 client.15333 trim_cache size 0 max 16384 2024-04-26T21:33:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 Processor -- start 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 -- start start 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7582cb0 0x55cbd7593bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7598180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbd7598ce0 con 0x55cbd75822c0 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbd75993b0 con 0x55cbd7587e20 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.067+0000 7f5ddd028740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbd7599a80 con 0x55cbd7582cb0 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44876/0 (socket says 172.21.15.97:44876) 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 -- 172.21.15.97:0/4165633757 learned_addr learned my addr 172.21.15.97:0/4165633757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7582cb0 0x55cbd7593bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3e00640 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7598180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 -- 172.21.15.97:0/4165633757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7582cb0 msgr2=0x55cbd7593bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7582cb0 0x55cbd7593bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 -- 172.21.15.97:0/4165633757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 msgr2=0x55cbd7598180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7598180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.068+0000 7f5dd3400640 1 -- 172.21.15.97:0/4165633757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd7579680 con 0x55cbd75822c0 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5dd3400640 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009230 tx=0x7f5dc4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5728d2a295976a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5dbf400640 1 -- 172.21.15.97:0/4165633757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4053070 con 0x55cbd75822c0 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5dbf400640 1 -- 172.21.15.97:0/4165633757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dc404edc0 con 0x55cbd75822c0 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5dbf400640 1 -- 172.21.15.97:0/4165633757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4040070 con 0x55cbd75822c0 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5ddd028740 1 -- 172.21.15.97:0/4165633757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 msgr2=0x55cbd758a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.069+0000 7f5ddd028740 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009230 tx=0x7f5dc4009260 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 -- 172.21.15.97:0/4165633757 shutdown_connections 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7598180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd7582cb0 0x55cbd7593bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 --2- 172.21.15.97:0/4165633757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd75822c0 0x55cbd758a210 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 -- 172.21.15.97:0/4165633757 >> 172.21.15.97:0/4165633757 conn(0x55cbd7516d10 msgr2=0x55cbd7584070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 -- 172.21.15.97:0/4165633757 shutdown_connections 2024-04-26T21:33:09.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 -- 172.21.15.97:0/4165633757 wait complete. 2024-04-26T21:33:09.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 -- start start 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.070+0000 7f5ddd028740 1 Processor -- start 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd758f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7592790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbd759a820 con 0x55cbd7582cb0 2024-04-26T21:33:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cbd759a150 con 0x55cbd7587e20 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cbd759aef0 con 0x55cbd75822c0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5ddd028740 10 client.0 _ll_register_callbacks cb 0x7f5dc0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44886/0 (socket says 172.21.15.97:44886) 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 -- 172.21.15.97:0/2179280571 learned_addr learned my addr 172.21.15.97:0/2179280571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd3400640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd758f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 -- 172.21.15.97:0/2179280571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 msgr2=0x55cbd758f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd758f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 -- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 msgr2=0x55cbd7592790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7592790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 -- 172.21.15.97:0/2179280571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd7592d80 con 0x55cbd7582cb0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dd2a00640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005f70 tx=0x7f5dc00109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ada96e5fe69aa75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.071+0000 7f5dbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0036020 con 0x55cbd7582cb0 2024-04-26T21:33:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f5dc0010dc0 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc404edc0 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dc0022070 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbd75ff4e0 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5dc404edc0 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dc001e450 con 0x55cbd7582cb0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 10 client.15339 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 20 client.15339 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7604100 0x55cbd7602e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 4 client.15339 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.072+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 --> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] -- command(tid 0: {"prefix": "status"}) -- 0x55cbd74ee080 con 0x55cbd7604100 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.073+0000 7f5dd3400640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7604100 0x55cbd7602e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.074+0000 7f5dd3400640 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7604100 0x55cbd7602e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=633a99832b8016e5 server_cookie=ec6c6df753a134b1 in_seq=0 out_seq=0 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.074+0000 7f5dbcc00640 10 client.15339 ms_handle_connect on v2:172.21.15.111:6836/1944730061 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5dbcc00640 1 -- 172.21.15.97:0/2179280571 <== mds.1 v2:172.21.15.111:6836/1944730061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbd74ee080 con 0x55cbd7604100 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5dbcc00640 10 client.15339 handle_command_reply: tid=0 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 client.15339 shutdown 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 msgr2=0x55cbd758f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005f70 tx=0x7f5dc00109c0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 shutdown_connections 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cbd7587e20 0x55cbd7592790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbd7582cb0 0x55cbd758f7e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.111:6836/1944730061,v1:172.21.15.111:6837/1944730061] conn(0x55cbd7604100 0x55cbd7602e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 --2- 172.21.15.97:0/2179280571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cbd75822c0 0x55cbd758f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5dbcc00640 0 client.15339 ms_handle_reset on v2:172.21.15.111:6836/1944730061 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 >> 172.21.15.97:0/2179280571 conn(0x55cbd7516d10 msgr2=0x55cbd7598180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 shutdown_connections 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 1 -- 172.21.15.97:0/2179280571 wait complete. 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.075+0000 7f5ddd028740 20 client.15339 trim_cache size 0 max 16384 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34460, 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:33:09.080 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 131.80049007900001, 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 157.875625877 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:09.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 msgr2=0x7f5db80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 0x7f5db80a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005b80 tx=0x7f5dc0005b10 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 msgr2=0x7f5dd41016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b970 tx=0x7f5dcc00be30 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 shutdown_connections 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f5db80a3c40 0x7f5db80a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd4108440 0x7f5dd41016d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dd4105b10 0x7f5dd4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 --2- 172.21.15.97:0/195716382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dd41031e0 0x7f5dd4102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 >> 172.21.15.97:0/195716382 conn(0x7f5dd40fcbf0 msgr2=0x7f5dd4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 shutdown_connections 2024-04-26T21:33:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.076+0000 7f5dd8c00640 1 -- 172.21.15.97:0/195716382 wait complete. 2024-04-26T21:33:09.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:33:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:08 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1716832227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.274+0000 7f3e50200640 1 -- 172.21.15.97:0/1916495964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48076970 msgr2=0x7f3e48074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.274+0000 7f3e50200640 1 --2- 172.21.15.97:0/1916495964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48076970 0x7f3e48074dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 -- 172.21.15.97:0/1916495964 shutdown_connections 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 --2- 172.21.15.97:0/1916495964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e4810a830 0x7f3e4810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 --2- 172.21.15.97:0/1916495964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e48075390 0x7f3e48075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 --2- 172.21.15.97:0/1916495964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48076970 0x7f3e48074dc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 -- 172.21.15.97:0/1916495964 >> 172.21.15.97:0/1916495964 conn(0x7f3e480fdee0 msgr2=0x7f3e48100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 -- 172.21.15.97:0/1916495964 shutdown_connections 2024-04-26T21:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.275+0000 7f3e50200640 1 -- 172.21.15.97:0/1916495964 wait complete. 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 Processor -- start 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 -- start start 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e48076970 0x7f3e4819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 0x7f3e481a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48110560 con 0x7f3e48075390 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481103e0 con 0x7f3e48076970 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e50200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481106e0 con 0x7f3e4810a830 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e4d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e4d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44892/0 (socket says 172.21.15.97:44892) 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.276+0000 7f3e4d800640 1 -- 172.21.15.97:0/1122020385 learned_addr learned my addr 172.21.15.97:0/1122020385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 0x7f3e481a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 -- 172.21.15.97:0/1122020385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 msgr2=0x7f3e481a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 0x7f3e481a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 -- 172.21.15.97:0/1122020385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e48076970 msgr2=0x7f3e4819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e48076970 0x7f3e4819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e48076970 0x7f3e4819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 -- 172.21.15.97:0/1122020385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481a2870 con 0x7f3e48075390 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 0x7f3e481a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e4d800640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c031460 tx=0x7f3e3c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e46000640 1 -- 172.21.15.97:0/1122020385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c03f070 con 0x7f3e48075390 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e46000640 1 -- 172.21.15.97:0/1122020385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e3c031da0 con 0x7f3e48075390 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e46000640 1 -- 172.21.15.97:0/1122020385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c035030 con 0x7f3e48075390 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.277+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481a2b00 con 0x7f3e48075390 2024-04-26T21:33:09.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.278+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a2ff0 con 0x7f3e48075390 2024-04-26T21:33:09.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 Processor -- start 2024-04-26T21:33:09.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 -- start start 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e660925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e66092c50 0x557e660950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e65ff1600 con 0x557e6608d0f0 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e65f8e100 con 0x557e66092c50 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e53071740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e65fed5c0 con 0x557e6608dae0 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44902/0 (socket says 172.21.15.97:44902) 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.281+0000 7f3e4e200640 1 -- 172.21.15.97:0/1499130179 learned_addr learned my addr 172.21.15.97:0/1499130179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e46000640 1 -- 172.21.15.97:0/1122020385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e3c042070 con 0x7f3e48075390 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e66092c50 0x557e660950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 -- 172.21.15.97:0/1499130179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 msgr2=0x557e660925e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e660925e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 -- 172.21.15.97:0/1499130179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e66092c50 msgr2=0x557e660950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e66092c50 0x557e660950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 -- 172.21.15.97:0/1499130179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e65d032d0 con 0x557e6608d0f0 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e40008dd0 tx=0x7f3e4000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b69c8f6cea6fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e3aa00640 1 -- 172.21.15.97:0/1499130179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000f830 con 0x557e6608d0f0 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e3aa00640 1 -- 172.21.15.97:0/1499130179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e4000de20 con 0x557e6608d0f0 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.282+0000 7f3e3aa00640 1 -- 172.21.15.97:0/1499130179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40018cf0 con 0x557e6608d0f0 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.283+0000 7f3e46000640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 0x7f3e300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.283+0000 7f3e53071740 1 -- 172.21.15.97:0/1499130179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 msgr2=0x557e6608d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.283+0000 7f3e53071740 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e40008dd0 tx=0x7f3e4000d710 comp rx=0 tx=0).stop 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e4d800640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 0x7f3e300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 -- 172.21.15.97:0/1499130179 shutdown_connections 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e66092c50 0x557e660950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e660925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 --2- 172.21.15.97:0/1499130179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e6608d510 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 -- 172.21.15.97:0/1499130179 >> 172.21.15.97:0/1499130179 conn(0x557e66021d10 msgr2=0x557e66088b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e46000640 1 -- 172.21.15.97:0/1122020385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3e3c0f0fd0 con 0x7f3e48075390 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 -- 172.21.15.97:0/1499130179 shutdown_connections 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 -- 172.21.15.97:0/1499130179 wait complete. 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e4d800640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 0x7f3e300a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x557e6608e550 tx=0x7f3e3c082040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 -- start start 2024-04-26T21:33:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.284+0000 7f3e53071740 1 Processor -- start 2024-04-26T21:33:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e66097580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e66097ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e66095c00 con 0x557e66092c50 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e65f78f70 con 0x557e6608d0f0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e66095d80 con 0x557e6608dae0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e53071740 10 client.0 _ll_register_callbacks cb 0x557e65fe7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44918/0 (socket says 172.21.15.97:44918) 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.285+0000 7f3e4e200640 1 -- 172.21.15.97:0/3444422668 learned_addr learned my addr 172.21.15.97:0/3444422668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e66097580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e66097ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 -- 172.21.15.97:0/3444422668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 msgr2=0x557e66097ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e66097ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 -- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 msgr2=0x557e66097580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e66097580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 -- 172.21.15.97:0/3444422668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e65dcd6b0 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e66097ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005fd0 tx=0x7f3e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f46e8d1b3a4df91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e2f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000fcb0 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3e40028400 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4000de20 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e40028600 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e53071740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.286+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e660a1b50 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e4000de20 con 0x557e66092c50 2024-04-26T21:33:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e4000e670 con 0x557e66092c50 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 10 client.15351 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 20 client.15351 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e660a3f30 0x557e660a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 4 client.15351 mds_command: new command op to 34502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.287+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 --> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e65fff210 con 0x557e660a3f30 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.288+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e660a3f30 0x557e660a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.289+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e660a3f30 0x557e660a53a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3e7ccc6def258 server_cookie=759505fc3f25b158 in_seq=0 out_seq=0 2024-04-26T21:33:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.289+0000 7f3e2f400640 10 client.15351 ms_handle_connect on v2:172.21.15.111:6838/2198422773 2024-04-26T21:33:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e2f400640 1 -- 172.21.15.97:0/3444422668 <== mds.0 v2:172.21.15.111:6838/2198422773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e65fff210 con 0x557e660a3f30 2024-04-26T21:33:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e2f400640 10 client.15351 handle_command_reply: tid=0 2024-04-26T21:33:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 client.15351 shutdown 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 msgr2=0x557e6609eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005fd0 tx=0x7f3e40004a40 comp rx=0 tx=0).stop 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 shutdown_connections 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e660a3f30 0x557e660a53a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609eb00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e66097ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 --2- 172.21.15.97:0/3444422668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e66097580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e2f400640 0 client.15351 ms_handle_reset on v2:172.21.15.111:6838/2198422773 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 >> 172.21.15.97:0/3444422668 conn(0x557e66021d10 msgr2=0x557e66087eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 shutdown_connections 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.291+0000 7f3e53071740 1 -- 172.21.15.97:0/3444422668 wait complete. 2024-04-26T21:33:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.292+0000 7f3e53071740 20 client.15351 trim_cache size 0 max 16384 2024-04-26T21:33:09.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:08 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1716832227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 Processor -- start 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 -- start start 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608dae0 0x557e6609eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e66092c50 0x557e66087620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e660a36e0 con 0x557e6608d0f0 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e660a3db0 con 0x557e6608dae0 2024-04-26T21:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e53071740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e660a4480 con 0x557e66092c50 2024-04-26T21:33:09.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44920/0 (socket says 172.21.15.97:44920) 2024-04-26T21:33:09.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.312+0000 7f3e4e200640 1 -- 172.21.15.97:0/3833630664 learned_addr learned my addr 172.21.15.97:0/3833630664 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608dae0 0x557e6609eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 -- 172.21.15.97:0/3833630664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e66092c50 msgr2=0x557e66087620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e66092c50 0x557e66087620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 -- 172.21.15.97:0/3833630664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608dae0 msgr2=0x557e6609eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608dae0 0x557e6609eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 -- 172.21.15.97:0/3833630664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6608dec0 con 0x557e6608d0f0 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e4e200640 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e40008250 tx=0x7f3e400085b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb4daf9f858ff34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e3a000640 1 -- 172.21.15.97:0/3833630664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40037070 con 0x557e6608d0f0 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e3a000640 1 -- 172.21.15.97:0/3833630664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e40028400 con 0x557e6608d0f0 2024-04-26T21:33:09.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.313+0000 7f3e3a000640 1 -- 172.21.15.97:0/3833630664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40018780 con 0x557e6608d0f0 2024-04-26T21:33:09.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 -- 172.21.15.97:0/3833630664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 msgr2=0x557e66097a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e40008250 tx=0x7f3e400085b0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 -- 172.21.15.97:0/3833630664 shutdown_connections 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e66092c50 0x557e66087620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608dae0 0x557e6609eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 --2- 172.21.15.97:0/3833630664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6608d0f0 0x557e66097a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 -- 172.21.15.97:0/3833630664 >> 172.21.15.97:0/3833630664 conn(0x557e66021d10 msgr2=0x557e660a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 -- 172.21.15.97:0/3833630664 shutdown_connections 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.314+0000 7f3e53071740 1 -- 172.21.15.97:0/3833630664 wait complete. 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 -- start start 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 Processor -- start 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e6609acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e6609b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e660a5220 con 0x557e66092c50 2024-04-26T21:33:09.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e660a4b50 con 0x557e6608d0f0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e660a58f0 con 0x557e6608dae0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.315+0000 7f3e53071740 10 client.0 _ll_register_callbacks cb 0x7f3e40008c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44934/0 (socket says 172.21.15.97:44934) 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 -- 172.21.15.97:0/1837649071 learned_addr learned my addr 172.21.15.97:0/1837649071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e6609acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e6609b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 -- 172.21.15.97:0/1837649071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 msgr2=0x557e6609b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e6609b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 -- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 msgr2=0x557e6609acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e6609acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 -- 172.21.15.97:0/1837649071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6609e6e0 con 0x557e66092c50 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e4e200640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e40018a60 tx=0x7f3e40018da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29159886ed4bd977 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:09.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e2f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.316+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40043070 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f3e40018780 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e40028400 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e40037040 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e6610a7e0 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e40028400 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e4003c5e0 con 0x557e66092c50 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 20 client.15357 populate_metadata read hostname 'smithi097' 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e6610f050 0x557e6610bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 4 client.15357 mds_command: new command op to 34502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.317+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 --> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] -- command(tid 0: {"prefix": "status"}) -- 0x557e65ff9080 con 0x557e6610f050 2024-04-26T21:33:09.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.318+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e6610f050 0x557e6610bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.319+0000 7f3e4ce00640 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e6610f050 0x557e6610bb60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5540d50ed41b23 server_cookie=bc93987489b07dcc in_seq=0 out_seq=0 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.319+0000 7f3e2f400640 10 client.15357 ms_handle_connect on v2:172.21.15.111:6838/2198422773 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.319+0000 7f3e2f400640 1 -- 172.21.15.97:0/1837649071 <== mds.0 v2:172.21.15.111:6838/2198422773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557e65ff9080 con 0x557e6610f050 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.319+0000 7f3e2f400640 10 client.15357 handle_command_reply: tid=0 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.319+0000 7f3e53071740 1 client.15357 shutdown 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 msgr2=0x557e6609e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e40018a60 tx=0x7f3e40018da0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 shutdown_connections 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:6838/2198422773,v1:172.21.15.111:6839/2198422773] conn(0x557e6610f050 0x557e6610bb60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e66092c50 0x557e6609e0f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6608dae0 0x557e6609b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e2f400640 0 client.15357 ms_handle_reset on v2:172.21.15.111:6838/2198422773 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 --2- 172.21.15.97:0/1837649071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e6608d0f0 0x557e6609acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 >> 172.21.15.97:0/1837649071 conn(0x557e66021d10 msgr2=0x557e6608bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 shutdown_connections 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 1 -- 172.21.15.97:0/1837649071 wait complete. 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.320+0000 7f3e53071740 20 client.15357 trim_cache size 0 max 16384 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9ce681f8-0412-11ef-bc93-c7b262605968", 2024-04-26T21:33:09.324 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34502, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 135.36434328199999, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 90, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 142.18874652900001 2024-04-26T21:33:09.325 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 msgr2=0x7f3e300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 0x7f3e300a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x557e6608e550 tx=0x7f3e3c082040 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 msgr2=0x7f3e4819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c031460 tx=0x7f3e3c003d50 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 shutdown_connections 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e4810a830 0x7f3e481a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e48076970 0x7f3e4819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f3e300a3c40 0x7f3e300a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 --2- 172.21.15.97:0/1122020385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e48075390 0x7f3e4819ac60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:09.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 >> 172.21.15.97:0/1122020385 conn(0x7f3e480fdee0 msgr2=0x7f3e48106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:09.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 shutdown_connections 2024-04-26T21:33:09.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:09.321+0000 7f3e50200640 1 -- 172.21.15.97:0/1122020385 wait complete. 2024-04-26T21:33:09.340 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:33:09.353 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:33:09.353 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:33:09.354 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:33:09.354 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T21:33:09.413 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T21:33:09.413 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-26T21:33:09.413 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:33:09.413 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:33:09.413 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:09.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:09.413 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-26T21:33:09.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:09.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-26T21:33:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-04-26T21:33:09.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:33:09.562 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T21:33:09.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:33:10.005 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:33:10.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:09 smithi097 ceph-mon[28127]: pgmap v434: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:09 smithi111 ceph-mon[31001]: pgmap v434: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:09 smithi143 ceph-mon[31461]: pgmap v434: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:12.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:11 smithi097 ceph-mon[28127]: pgmap v435: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:11 smithi111 ceph-mon[31001]: pgmap v435: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 -- 172.21.15.97:0/3732660356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 msgr2=0x7faa98105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 --2- 172.21.15.97:0/3732660356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa98105ab0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faa84009990 tx=0x7faa84009ca0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 -- 172.21.15.97:0/3732660356 shutdown_connections 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 --2- 172.21.15.97:0/3732660356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 0x7faa98112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 --2- 172.21.15.97:0/3732660356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 0x7faa9810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 --2- 172.21.15.97:0/3732660356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa98105ab0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 -- 172.21.15.97:0/3732660356 >> 172.21.15.97:0/3732660356 conn(0x7faa98101100 msgr2=0x7faa98103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 -- 172.21.15.97:0/3732660356 shutdown_connections 2024-04-26T21:33:12.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.284+0000 7faa9da00700 1 -- 172.21.15.97:0/3732660356 wait complete. 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.285+0000 7faa9da00700 1 Processor -- start 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 -- start start 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 0x7faa981a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 0x7faa9819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa98115720 con 0x7faa98105630 2024-04-26T21:33:12.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa981155a0 con 0x7faa98105ff0 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa9da00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faa981158a0 con 0x7faa9810b630 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33892/0 (socket says 172.21.15.97:33892) 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.286+0000 7faa96a00700 1 -- 172.21.15.97:0/1240676139 learned_addr learned my addr 172.21.15.97:0/1240676139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96000700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 0x7faa981a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa97400700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 0x7faa9819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 -- 172.21.15.97:0/1240676139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 msgr2=0x7faa9819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 0x7faa9819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 -- 172.21.15.97:0/1240676139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 msgr2=0x7faa981a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 0x7faa981a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 -- 172.21.15.97:0/1240676139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9819c1a0 con 0x7faa98105630 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa96a00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faa8800d980 tx=0x7faa8800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:12.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88017070 con 0x7faa98105630 2024-04-26T21:33:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa88004830 con 0x7faa98105630 2024-04-26T21:33:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.287+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9819c490 con 0x7faa98105630 2024-04-26T21:33:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.288+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880053a0 con 0x7faa98105630 2024-04-26T21:33:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.288+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981a8700 con 0x7faa98105630 2024-04-26T21:33:12.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.289+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9806a110 con 0x7faa98105630 2024-04-26T21:33:12.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.291+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faa880049e0 con 0x7faa98105630 2024-04-26T21:33:12.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.293+0000 7faa8f400700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 0x7faa7c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:12.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.293+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7faa880d07e0 con 0x7faa98105630 2024-04-26T21:33:12.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.293+0000 7faa96000700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 0x7faa7c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:11 smithi143 ceph-mon[31461]: pgmap v435: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:12.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.294+0000 7faa96000700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 0x7faa7c0a6210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa84005850 tx=0x7faa8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:12.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.296+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faa88020020 con 0x7faa98105630 2024-04-26T21:33:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.573+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa9802e490 con 0x7faa98105630 2024-04-26T21:33:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.574+0000 7faa8f400700 1 -- 172.21.15.97:0/1240676139 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7faa8809d0f0 con 0x7faa98105630 2024-04-26T21:33:12.575 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:33:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 msgr2=0x7faa7c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 0x7faa7c0a6210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa84005850 tx=0x7faa8401a040 comp rx=0 tx=0).stop 2024-04-26T21:33:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 msgr2=0x7faa981a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faa8800d980 tx=0x7faa8800dd40 comp rx=0 tx=0).stop 2024-04-26T21:33:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 shutdown_connections 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7faa7c0a3d50 0x7faa7c0a6210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa9810b630 0x7faa9819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa98105630 0x7faa981a1830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 --2- 172.21.15.97:0/1240676139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa98105ff0 0x7faa981a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.578+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 >> 172.21.15.97:0/1240676139 conn(0x7faa98101100 msgr2=0x7faa981018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.579+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 shutdown_connections 2024-04-26T21:33:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:12.579+0000 7faa9da00700 1 -- 172.21.15.97:0/1240676139 wait complete. 2024-04-26T21:33:13.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:12 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1240676139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:33:13.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:12 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1240676139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:33:13.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:12 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1240676139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:13.326 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:33:13.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:33:13.327 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:33:13.327 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:33:13.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:13.327 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:33:13.379 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41866sec preferred_lft 41866sec 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-04-26T21:33:13.380 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:33:13.381 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:33:13.381 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-26T21:33:13.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:33:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:13.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:13.836 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:33:13.901 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:13 smithi097 ceph-mon[28127]: pgmap v436: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:13.908 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-26T21:33:13.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:13.911 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-26T21:33:13.964 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-26T21:33:13.964 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:33:13.964 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:33:13.964 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:33:13.964 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:33:13.965 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-26T21:33:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:33:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:13 smithi111 ceph-mon[31001]: pgmap v436: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:13 smithi143 ceph-mon[31461]: pgmap v436: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:14.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:14.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:14.690 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-26T21:33:14.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:14.742 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-04-26T21:33:14.794 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:33:14.794 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-26T21:33:14.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:14.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:33:14.794 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-26T21:33:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:33:14.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:14.977 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:33:14.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:33:14.978 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-26T21:33:15.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:33:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:15.272 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:33:15.272 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-26T21:33:15.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-26T21:33:15.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:33:15.272 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-26T21:33:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:33:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:15.470 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:33:15.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:33:15.470 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:15.520 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:33:15.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:33:15.521 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:15.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-26T21:33:15.695 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-26T21:33:15.753 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:33:15.754 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/19df09460c61c02cc942d4377aaae84e243295b38bb15dcebf97c60a7032d46f/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/560e446b79482aed787bd07e2f8ebbc2d9a30d6e6b695fc12fb88d3d1b09810b/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/c14ebd522dfcbc47b96280d1add6bfb27c905d1a6fa388213d4f33e008e59e80/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/aeba0ca814bfa7b92a3e62715ec27512ceca88adf305e73b09d8aa02b4f56c5a/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1d8ac2b9d02d0f356553344d83b0fe3def0205a49e9ceaa5b8c908cedfece76e/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/e2a13e333d4a4b82076ee6d16f853bb51680c07400e8e92d29f350f889e12b87/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/057520bd00f538ae5e0e4f179fedc48b06e3a77c6e19d5217f0d06d510b01aac/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/20f48d4d93408f0a14bec2491186b2dddd885cbeb9b7f65caa4f121d37726d01/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1e33163917fdb2e4cfb08fe31161541a118e8e89142007e28055b3ecf6d326af/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/651482dfe6ca73f424c8c14ecb1e2215aa1533b926f60ace6e600408af0210ec/merged 2024-04-26T21:33:15.755 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-26T21:33:15.756 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:15.756 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:15.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:15.757 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-26T21:33:15.807 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:33:15.807 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/a7f60f78-af7e-4156-a7e8-d4c7197c1a4e --client_fs=cephfs) 2024-04-26T21:33:15.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-26T21:33:15.881 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-26T21:33:15.923 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-26T21:33:15.924 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/19df09460c61c02cc942d4377aaae84e243295b38bb15dcebf97c60a7032d46f/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/560e446b79482aed787bd07e2f8ebbc2d9a30d6e6b695fc12fb88d3d1b09810b/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/c14ebd522dfcbc47b96280d1add6bfb27c905d1a6fa388213d4f33e008e59e80/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/aeba0ca814bfa7b92a3e62715ec27512ceca88adf305e73b09d8aa02b4f56c5a/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1d8ac2b9d02d0f356553344d83b0fe3def0205a49e9ceaa5b8c908cedfece76e/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/e2a13e333d4a4b82076ee6d16f853bb51680c07400e8e92d29f350f889e12b87/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/057520bd00f538ae5e0e4f179fedc48b06e3a77c6e19d5217f0d06d510b01aac/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/20f48d4d93408f0a14bec2491186b2dddd885cbeb9b7f65caa4f121d37726d01/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1e33163917fdb2e4cfb08fe31161541a118e8e89142007e28055b3ecf6d326af/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/651482dfe6ca73f424c8c14ecb1e2215aa1533b926f60ace6e600408af0210ec/merged 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:15.925 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:15.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:15.927 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-26T21:33:16.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-04-26T21:33:16.015+0000 7fe72b046480 -1 init, newargv = 0x558c35ecd650 newargc=15 2024-04-26T21:33:16.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[80356]: starting ceph client 2024-04-26T21:33:16.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[80356]: starting fuse 2024-04-26T21:33:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:15 smithi111 ceph-mon[31001]: pgmap v437: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:16.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:15 smithi143 ceph-mon[31461]: pgmap v437: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:16.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:15 smithi097 ceph-mon[28127]: pgmap v437: 97 pgs: 97 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:17.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-04-26T21:33:17.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:16 smithi097 ceph-mon[28127]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:33:17.203 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-04-26T21:33:17.256 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:33:17.257 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/19df09460c61c02cc942d4377aaae84e243295b38bb15dcebf97c60a7032d46f/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/560e446b79482aed787bd07e2f8ebbc2d9a30d6e6b695fc12fb88d3d1b09810b/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/c14ebd522dfcbc47b96280d1add6bfb27c905d1a6fa388213d4f33e008e59e80/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/aeba0ca814bfa7b92a3e62715ec27512ceca88adf305e73b09d8aa02b4f56c5a/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1d8ac2b9d02d0f356553344d83b0fe3def0205a49e9ceaa5b8c908cedfece76e/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/e2a13e333d4a4b82076ee6d16f853bb51680c07400e8e92d29f350f889e12b87/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/057520bd00f538ae5e0e4f179fedc48b06e3a77c6e19d5217f0d06d510b01aac/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/20f48d4d93408f0a14bec2491186b2dddd885cbeb9b7f65caa4f121d37726d01/merged 2024-04-26T21:33:17.258 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/1e33163917fdb2e4cfb08fe31161541a118e8e89142007e28055b3ecf6d326af/merged 2024-04-26T21:33:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/651482dfe6ca73f424c8c14ecb1e2215aa1533b926f60ace6e600408af0210ec/merged 2024-04-26T21:33:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-04-26T21:33:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:33:17.259 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:17.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:17.260 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-04-26T21:33:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:16 smithi143 ceph-mon[31461]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:33:17.308 INFO:teuthology.orchestra.run.smithi097.stdout:114 2024-04-26T21:33:17.308 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-04-26T21:33:17.308 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-04-26T21:33:17.309 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-04-26T21:33:17.310 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-04-26T21:33:17.310 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:33:17.310 DEBUG:teuthology.orchestra.run.smithi097:> 2024-04-26T21:33:17.310 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-04-26T21:33:17.310 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-04-26T21:33:17.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:16 smithi111 ceph-mon[31001]: pgmap v438: 97 pgs: 97 active+clean; 636 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:33:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.80356.asok 2024-04-26T21:33:17.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:33:17.482 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80356.asok 2024-04-26T21:33:17.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:17.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80356.asok status 2024-04-26T21:33:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "80356", 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/volumes/qa/sv_0/a7f60f78-af7e-4156-a7e8-d4c7197c1a4e" 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 15369, 2024-04-26T21:33:17.708 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 15369 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3605843854 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:33:17.709 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3605843854 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.15369 192.168.0.1:0/3605843854", 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/3605843854", 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 5, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 26, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 90, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-04-26T21:33:17.710 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T21:33:17.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:17.722 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-04-26T21:33:17.722 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:17.776 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-04-26T21:33:17.777 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:17.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:33:17.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:33:17.875 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T21:33:17.886 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:33:17.886 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:33:17.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-26T21:33:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 -- 172.21.15.97:0/987586561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108108550 msgr2=0x7f110810a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 --2- 172.21.15.97:0/987586561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108108550 0x7f110810a940 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1100009ca0 tx=0x7f110002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 -- 172.21.15.97:0/987586561 shutdown_connections 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 --2- 172.21.15.97:0/987586561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108108550 0x7f110810a940 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 --2- 172.21.15.97:0/987586561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1108105c20 0x7f1108108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 --2- 172.21.15.97:0/987586561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11081032f0 0x7f11081056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 -- 172.21.15.97:0/987586561 >> 172.21.15.97:0/987586561 conn(0x7f11080fcc20 msgr2=0x7f11080ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 -- 172.21.15.97:0/987586561 shutdown_connections 2024-04-26T21:33:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.084+0000 7f110ea00640 1 -- 172.21.15.97:0/987586561 wait complete. 2024-04-26T21:33:18.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.085+0000 7f110ea00640 1 Processor -- start 2024-04-26T21:33:18.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.085+0000 7f110ea00640 1 -- start start 2024-04-26T21:33:18.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11081032f0 0x7f110819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 0x7f11081a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f110810e210 con 0x7f1108105c20 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f110810e090 con 0x7f1108108550 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110ea00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f110810e390 con 0x7f11081032f0 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33912/0 (socket says 172.21.15.97:33912) 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 -- 172.21.15.97:0/2236422392 learned_addr learned my addr 172.21.15.97:0/2236422392 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110e000640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 0x7f11081a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110d600640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11081032f0 0x7f110819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 -- 172.21.15.97:0/2236422392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11081032f0 msgr2=0x7f110819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11081032f0 0x7f110819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 -- 172.21.15.97:0/2236422392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 msgr2=0x7f11081a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 0x7f11081a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110cc00640 1 -- 172.21.15.97:0/2236422392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a2700 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.086+0000 7f110e000640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 0x7f11081a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f110cc00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c970 tx=0x7f10f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4007bb0 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10f4007d50 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4018660 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a29f0 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.087+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a2ef0 con 0x7f1108105c20 2024-04-26T21:33:18.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.088+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110810a040 con 0x7f1108105c20 2024-04-26T21:33:18.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.090+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10f400f970 con 0x7f1108105c20 2024-04-26T21:33:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.092+0000 7f1106000640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 0x7f10f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.093+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f10f40d06d0 con 0x7f1108105c20 2024-04-26T21:33:18.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.093+0000 7f110d600640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 0x7f10f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.093+0000 7f110d600640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 0x7f10f00a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10fc005fd0 tx=0x7f10fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:18.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.097+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f10f4014030 con 0x7f1108105c20 2024-04-26T21:33:18.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.346+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f110802dd00 con 0x7f1108105c20 2024-04-26T21:33:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.401+0000 7f1106000640 1 -- 172.21.15.97:0/2236422392 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f10f409c910 con 0x7f1108105c20 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 msgr2=0x7f10f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 0x7f10f00a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10fc005fd0 tx=0x7f10fc005850 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 msgr2=0x7f110819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c970 tx=0x7f10f400ce30 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 shutdown_connections 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1108108550 0x7f11081a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10f00a3c30 0x7f10f00a60f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108105c20 0x7f110819b020 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 --2- 172.21.15.97:0/2236422392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11081032f0 0x7f110819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 >> 172.21.15.97:0/2236422392 conn(0x7f11080fcc20 msgr2=0x7f1108102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 shutdown_connections 2024-04-26T21:33:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.405+0000 7f110ea00640 1 -- 172.21.15.97:0/2236422392 wait complete. 2024-04-26T21:33:18.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 -- 172.21.15.97:0/4177499309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0105db0 msgr2=0x7fa4d0106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 --2- 172.21.15.97:0/4177499309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0105db0 0x7fa4d0106230 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009900 tx=0x7fa4b802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 -- 172.21.15.97:0/4177499309 shutdown_connections 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 --2- 172.21.15.97:0/4177499309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d0106770 0x7fa4d010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 --2- 172.21.15.97:0/4177499309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0105db0 0x7fa4d0106230 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 --2- 172.21.15.97:0/4177499309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4d0104bc0 0x7fa4d0104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 -- 172.21.15.97:0/4177499309 >> 172.21.15.97:0/4177499309 conn(0x7fa4d0100150 msgr2=0x7fa4d0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:33:18.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 -- 172.21.15.97:0/4177499309 shutdown_connections 2024-04-26T21:33:18.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.624+0000 7fa4d5600640 1 -- 172.21.15.97:0/4177499309 wait complete. 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 Processor -- start 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 -- start start 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0104bc0 0x7fa4d019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d0105db0 0x7fa4d019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4d0106770 0x7fa4d01a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0110580 con 0x7fa4d0104bc0 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0110400 con 0x7fa4d0105db0 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.625+0000 7fa4d5600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0110700 con 0x7fa4d0106770 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0104bc0 0x7fa4d019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0104bc0 0x7fa4d019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33940/0 (socket says 172.21.15.97:33940) 2024-04-26T21:33:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 -- 172.21.15.97:0/2795231492 learned_addr learned my addr 172.21.15.97:0/2795231492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cf400640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4d0106770 0x7fa4d01a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 -- 172.21.15.97:0/2795231492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4d0106770 msgr2=0x7fa4d01a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d0105db0 0x7fa4d019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4d0106770 0x7fa4d01a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 -- 172.21.15.97:0/2795231492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d0105db0 msgr2=0x7fa4d019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d0105db0 0x7fa4d019b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d01a27c0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.626+0000 7fa4cea00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d0104bc0 0x7fa4d019aba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000c970 tx=0x7fa4c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.627+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0007c10 con 0x7fa4d0104bc0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.627+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4c0007db0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.627+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000f7d0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.627+0000 7fa4d5600640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d01a2ab0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.627+0000 7fa4d5600640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d01a2fb0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.628+0000 7fa4d5600640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d010be80 con 0x7fa4d0104bc0 2024-04-26T21:33:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.630+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa4c0018480 con 0x7fa4d0104bc0 2024-04-26T21:33:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.631+0000 7fa4c7e00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 0x7fa4b40a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:33:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.632+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fa4c00d09a0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.632+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 0x7fa4b40a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:33:18.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.633+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 0x7fa4b40a61b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8031460 tx=0x7fa4b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:33:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.636+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa4c009d2c0 con 0x7fa4d0104bc0 2024-04-26T21:33:18.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:18 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T21:33:18.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:18 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T21:33:18.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:33:18.872+0000 7fa4d5600640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa4d0062e10 con 0x7fa4b40a3cf0 2024-04-26T21:33:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:18 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T21:33:19.717 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:19 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T21:33:19.717 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:19 smithi097 ceph-mon[28127]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:33:19.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:19 smithi097 ceph-mon[28127]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-26T21:33:19.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:19 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T21:33:19.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:19 smithi143 ceph-mon[31461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:33:19.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:19 smithi143 ceph-mon[31461]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-26T21:33:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:19 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2236422392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T21:33:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:19 smithi111 ceph-mon[31001]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:33:19.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:19 smithi111 ceph-mon[31001]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-26T21:33:20.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:20 smithi143 ceph-mon[31461]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:33:20.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:20 smithi097 ceph-mon[28127]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:33:20.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:20 smithi111 ceph-mon[31001]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:33:21.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:21 smithi143 ceph-mon[31461]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:21.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:21 smithi097 ceph-mon[28127]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:21.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:21 smithi111 ceph-mon[31001]: pgmap v440: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:22.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:22 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:23.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:22 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:23.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:22 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:23 smithi111 ceph-mon[31001]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:33:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:23.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:23 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:23.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:23 smithi097 ceph-mon[28127]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:33:23.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:23.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:23 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:24.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:23 smithi143 ceph-mon[31461]: pgmap v441: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:33:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:24.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:23 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:25 smithi143 ceph-mon[31461]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:26.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:25 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:25 smithi097 ceph-mon[28127]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:25 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:25 smithi111 ceph-mon[31001]: pgmap v442: 97 pgs: 97 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-26T21:33:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:26.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:25 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:27 smithi111 ceph-mon[31001]: pgmap v443: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:33:27.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:27 smithi143 ceph-mon[31461]: pgmap v443: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:33:27.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:27.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:27 smithi097 ceph-mon[28127]: pgmap v443: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:33:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:28 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:28.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:28.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:28.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:28 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:28.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:28.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:28.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:28 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:33:29.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:29 smithi111 ceph-mon[31001]: pgmap v444: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:33:29.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:29 smithi143 ceph-mon[31461]: pgmap v444: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:33:29.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:29 smithi097 ceph-mon[28127]: pgmap v444: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:33:32.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:31 smithi143 ceph-mon[31461]: pgmap v445: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-26T21:33:32.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:31 smithi097 ceph-mon[28127]: pgmap v445: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-26T21:33:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:31 smithi111 ceph-mon[31001]: pgmap v445: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-26T21:33:34.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:33 smithi143 ceph-mon[31461]: pgmap v446: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:33:34.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:33 smithi097 ceph-mon[28127]: pgmap v446: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:33:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:33 smithi111 ceph-mon[31001]: pgmap v446: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:33:36.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:35 smithi143 ceph-mon[31461]: pgmap v447: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-26T21:33:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:35 smithi097 ceph-mon[28127]: pgmap v447: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-26T21:33:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:35 smithi111 ceph-mon[31001]: pgmap v447: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-26T21:33:38.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:37 smithi143 ceph-mon[31461]: pgmap v448: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-26T21:33:38.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:37 smithi097 ceph-mon[28127]: pgmap v448: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-26T21:33:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:37 smithi111 ceph-mon[31001]: pgmap v448: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-26T21:33:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:39 smithi097 ceph-mon[28127]: pgmap v449: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:39 smithi111 ceph-mon[31001]: pgmap v449: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:39 smithi143 ceph-mon[31461]: pgmap v449: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:42.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:41 smithi097 ceph-mon[28127]: pgmap v450: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:41 smithi111 ceph-mon[31001]: pgmap v450: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:41 smithi143 ceph-mon[31461]: pgmap v450: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:43 smithi111 ceph-mon[31001]: pgmap v451: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:44.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:43 smithi097 ceph-mon[28127]: pgmap v451: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:43 smithi143 ceph-mon[31461]: pgmap v451: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:33:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:45 smithi111 ceph-mon[31001]: pgmap v452: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:45 smithi143 ceph-mon[31461]: pgmap v452: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:46.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:45 smithi097 ceph-mon[28127]: pgmap v452: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:33:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:47 smithi111 ceph-mon[31001]: pgmap v453: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:33:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:47 smithi143 ceph-mon[31461]: pgmap v453: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:33:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:47 smithi097 ceph-mon[28127]: pgmap v453: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:33:50.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:49 smithi097 ceph-mon[28127]: pgmap v454: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:49 smithi111 ceph-mon[31001]: pgmap v454: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:50.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:49 smithi143 ceph-mon[31461]: pgmap v454: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:51 smithi111 ceph-mon[31001]: pgmap v455: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:33:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:51 smithi143 ceph-mon[31461]: pgmap v455: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:33:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:51 smithi097 ceph-mon[28127]: pgmap v455: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:33:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:53 smithi111 ceph-mon[31001]: pgmap v456: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:33:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:53 smithi143 ceph-mon[31461]: pgmap v456: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:33:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:53 smithi097 ceph-mon[28127]: pgmap v456: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:33:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:55 smithi111 ceph-mon[31001]: pgmap v457: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:56.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:55 smithi143 ceph-mon[31461]: pgmap v457: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:56.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:55 smithi097 ceph-mon[28127]: pgmap v457: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:33:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:57 smithi111 ceph-mon[31001]: pgmap v458: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:33:58.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:57 smithi143 ceph-mon[31461]: pgmap v458: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:33:58.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:57 smithi097 ceph-mon[28127]: pgmap v458: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-26T21:34:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:33:59 smithi111 ceph-mon[31001]: pgmap v459: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:00.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:33:59 smithi143 ceph-mon[31461]: pgmap v459: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:00.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:33:59 smithi097 ceph-mon[28127]: pgmap v459: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:01 smithi111 ceph-mon[31001]: pgmap v460: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:02.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:01 smithi097 ceph-mon[28127]: pgmap v460: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:02.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:01 smithi143 ceph-mon[31461]: pgmap v460: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:03 smithi111 ceph-mon[31001]: pgmap v461: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:04.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:03 smithi143 ceph-mon[31461]: pgmap v461: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:04.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:03 smithi097 ceph-mon[28127]: pgmap v461: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:04 smithi143 ceph-mon[31461]: pgmap v462: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:04 smithi097 ceph-mon[28127]: pgmap v462: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:05.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:04 smithi111 ceph-mon[31001]: pgmap v462: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:08.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:07 smithi143 ceph-mon[31461]: pgmap v463: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:08.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:07 smithi097 ceph-mon[28127]: pgmap v463: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:07 smithi111 ceph-mon[31001]: pgmap v463: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:10.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:09 smithi097 ceph-mon[28127]: pgmap v464: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:09 smithi111 ceph-mon[31001]: pgmap v464: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:09 smithi143 ceph-mon[31461]: pgmap v464: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:12.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:11 smithi097 ceph-mon[28127]: pgmap v465: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:11 smithi111 ceph-mon[31001]: pgmap v465: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:11 smithi143 ceph-mon[31461]: pgmap v465: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:14.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:13 smithi097 ceph-mon[28127]: pgmap v466: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:34:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:13 smithi111 ceph-mon[31001]: pgmap v466: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:34:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:13 smithi143 ceph-mon[31461]: pgmap v466: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:34:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:15 smithi111 ceph-mon[31001]: pgmap v467: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:16.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:15 smithi143 ceph-mon[31461]: pgmap v467: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:15 smithi097 ceph-mon[28127]: pgmap v467: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:17 smithi111 ceph-mon[31001]: pgmap v468: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:34:18.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:17 smithi143 ceph-mon[31461]: pgmap v468: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:34:18.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:17 smithi097 ceph-mon[28127]: pgmap v468: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:34:20.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:19 smithi097 ceph-mon[28127]: pgmap v469: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:19 smithi111 ceph-mon[31001]: pgmap v469: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:19 smithi143 ceph-mon[31461]: pgmap v469: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:21 smithi111 ceph-mon[31001]: pgmap v470: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:21 smithi143 ceph-mon[31461]: pgmap v470: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:22.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:21 smithi097 ceph-mon[28127]: pgmap v470: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:34:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:23 smithi111 ceph-mon[31001]: pgmap v471: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:23 smithi143 ceph-mon[31461]: pgmap v471: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:24.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:23 smithi097 ceph-mon[28127]: pgmap v471: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:25 smithi111 ceph-mon[31001]: pgmap v472: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:25 smithi143 ceph-mon[31461]: pgmap v472: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:25 smithi097 ceph-mon[28127]: pgmap v472: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:27 smithi111 ceph-mon[31001]: pgmap v473: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:27 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:28.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:27 smithi143 ceph-mon[31461]: pgmap v473: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:28.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:27 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:27 smithi097 ceph-mon[28127]: pgmap v473: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:27 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:29 smithi111 ceph-mon[31001]: pgmap v474: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:34:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:29 smithi143 ceph-mon[31461]: pgmap v474: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:34:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:29 smithi097 ceph-mon[28127]: pgmap v474: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:34:32.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:31 smithi111 ceph-mon[31001]: pgmap v475: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:32.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:31 smithi143 ceph-mon[31461]: pgmap v475: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:32.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:31 smithi097 ceph-mon[28127]: pgmap v475: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:33 smithi111 ceph-mon[31001]: pgmap v476: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:34.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:33 smithi143 ceph-mon[31461]: pgmap v476: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:34.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:33 smithi097 ceph-mon[28127]: pgmap v476: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:36.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:35 smithi111 ceph-mon[31001]: pgmap v477: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:36.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:35 smithi143 ceph-mon[31461]: pgmap v477: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:36.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:35 smithi097 ceph-mon[28127]: pgmap v477: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:37.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:36 smithi143 ceph-mon[31461]: pgmap v478: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:37.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:36 smithi097 ceph-mon[28127]: pgmap v478: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:36 smithi111 ceph-mon[31001]: pgmap v478: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:40.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:39 smithi097 ceph-mon[28127]: pgmap v479: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:39 smithi111 ceph-mon[31001]: pgmap v479: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:39 smithi143 ceph-mon[31461]: pgmap v479: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:41 smithi097 ceph-mon[28127]: pgmap v480: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:41 smithi111 ceph-mon[31001]: pgmap v480: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:41 smithi143 ceph-mon[31461]: pgmap v480: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:43 smithi111 ceph-mon[31001]: pgmap v481: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:43 smithi143 ceph-mon[31461]: pgmap v481: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:44.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:43 smithi097 ceph-mon[28127]: pgmap v481: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:45 smithi111 ceph-mon[31001]: pgmap v482: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:45 smithi143 ceph-mon[31461]: pgmap v482: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:46.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:45 smithi097 ceph-mon[28127]: pgmap v482: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:47 smithi111 ceph-mon[31001]: pgmap v483: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:47 smithi143 ceph-mon[31461]: pgmap v483: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:47 smithi097 ceph-mon[28127]: pgmap v483: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:50.116 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:49 smithi097 ceph-mon[28127]: pgmap v484: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:49 smithi111 ceph-mon[31001]: pgmap v484: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:50.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:49 smithi143 ceph-mon[31461]: pgmap v484: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:51 smithi111 ceph-mon[31001]: pgmap v485: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:51 smithi143 ceph-mon[31461]: pgmap v485: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:51 smithi097 ceph-mon[28127]: pgmap v485: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:53 smithi111 ceph-mon[31001]: pgmap v486: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:53 smithi143 ceph-mon[31461]: pgmap v486: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:53 smithi097 ceph-mon[28127]: pgmap v486: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:34:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:55 smithi111 ceph-mon[31001]: pgmap v487: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:55 smithi143 ceph-mon[31461]: pgmap v487: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:56.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:55 smithi097 ceph-mon[28127]: pgmap v487: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:34:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:57 smithi111 ceph-mon[31001]: pgmap v488: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:58.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:57 smithi143 ceph-mon[31461]: pgmap v488: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:34:58.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:57 smithi097 ceph-mon[28127]: pgmap v488: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:00.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:34:59 smithi111 ceph-mon[31001]: pgmap v489: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:00.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:34:59 smithi143 ceph-mon[31461]: pgmap v489: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:00.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:34:59 smithi097 ceph-mon[28127]: pgmap v489: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:01 smithi111 ceph-mon[31001]: pgmap v490: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:02.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:01 smithi143 ceph-mon[31461]: pgmap v490: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:02.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:01 smithi097 ceph-mon[28127]: pgmap v490: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:03 smithi111 ceph-mon[31001]: pgmap v491: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:04.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:03 smithi143 ceph-mon[31461]: pgmap v491: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:04.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:03 smithi097 ceph-mon[28127]: pgmap v491: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:05 smithi111 ceph-mon[31001]: pgmap v492: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:06.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:05 smithi143 ceph-mon[31461]: pgmap v492: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:05 smithi097 ceph-mon[28127]: pgmap v492: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:07.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:06 smithi143 ceph-mon[31461]: pgmap v493: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:06 smithi097 ceph-mon[28127]: pgmap v493: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:07.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:06 smithi111 ceph-mon[31001]: pgmap v493: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:09 smithi111 ceph-mon[31001]: pgmap v494: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:09 smithi143 ceph-mon[31461]: pgmap v494: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:10.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:09 smithi097 ceph-mon[28127]: pgmap v494: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:11 smithi111 ceph-mon[31001]: pgmap v495: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:11 smithi143 ceph-mon[31461]: pgmap v495: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:12.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:11 smithi097 ceph-mon[28127]: pgmap v495: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:13 smithi111 ceph-mon[31001]: pgmap v496: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:13 smithi143 ceph-mon[31461]: pgmap v496: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:14.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:13 smithi097 ceph-mon[28127]: pgmap v496: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:16.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:15 smithi111 ceph-mon[31001]: pgmap v497: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:16.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:15 smithi143 ceph-mon[31461]: pgmap v497: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:16.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:15 smithi097 ceph-mon[28127]: pgmap v497: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:18.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:17 smithi111 ceph-mon[31001]: pgmap v498: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:18.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:17 smithi143 ceph-mon[31461]: pgmap v498: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:18.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:17 smithi097 ceph-mon[28127]: pgmap v498: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:20.116 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:19 smithi097 ceph-mon[28127]: pgmap v499: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:19 smithi111 ceph-mon[31001]: pgmap v499: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:19 smithi143 ceph-mon[31461]: pgmap v499: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:21 smithi111 ceph-mon[31001]: pgmap v500: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:21 smithi143 ceph-mon[31461]: pgmap v500: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:21 smithi097 ceph-mon[28127]: pgmap v500: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:23 smithi111 ceph-mon[31001]: pgmap v501: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:35:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:23 smithi143 ceph-mon[31461]: pgmap v501: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:35:24.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:23 smithi097 ceph-mon[28127]: pgmap v501: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:35:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:25 smithi111 ceph-mon[31001]: pgmap v502: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:25 smithi143 ceph-mon[31461]: pgmap v502: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:25 smithi097 ceph-mon[28127]: pgmap v502: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:27 smithi111 ceph-mon[31001]: pgmap v503: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:35:28.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:27 smithi143 ceph-mon[31461]: pgmap v503: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:35:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:27 smithi097 ceph-mon[28127]: pgmap v503: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:35:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:29 smithi111 ceph-mon[31001]: pgmap v504: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:29 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:29 smithi143 ceph-mon[31461]: pgmap v504: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:29 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:29 smithi097 ceph-mon[28127]: pgmap v504: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:35:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:29 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:31 smithi111 ceph-mon[31001]: pgmap v505: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:35:32.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:31 smithi143 ceph-mon[31461]: pgmap v505: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:32.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:35:32.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:31 smithi097 ceph-mon[28127]: pgmap v505: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:32.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:35:34.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:33 smithi143 ceph-mon[31461]: pgmap v506: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:34.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:33 smithi097 ceph-mon[28127]: pgmap v506: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:33 smithi111 ceph-mon[31001]: pgmap v506: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:34 smithi143 ceph-mon[31461]: pgmap v507: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:34 smithi097 ceph-mon[28127]: pgmap v507: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:34 smithi111 ceph-mon[31001]: pgmap v507: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:37 smithi111 ceph-mon[31001]: pgmap v508: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:37 smithi143 ceph-mon[31461]: pgmap v508: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:38.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:37 smithi097 ceph-mon[28127]: pgmap v508: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:39 smithi111 ceph-mon[31001]: pgmap v509: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:39 smithi143 ceph-mon[31461]: pgmap v509: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:40.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:39 smithi097 ceph-mon[28127]: pgmap v509: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:41 smithi111 ceph-mon[31001]: pgmap v510: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:41 smithi143 ceph-mon[31461]: pgmap v510: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:42.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:41 smithi097 ceph-mon[28127]: pgmap v510: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:43 smithi111 ceph-mon[31001]: pgmap v511: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:43 smithi143 ceph-mon[31461]: pgmap v511: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:44.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:43 smithi097 ceph-mon[28127]: pgmap v511: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:45 smithi111 ceph-mon[31001]: pgmap v512: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:45 smithi143 ceph-mon[31461]: pgmap v512: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:46.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:45 smithi097 ceph-mon[28127]: pgmap v512: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:47 smithi111 ceph-mon[31001]: pgmap v513: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:47 smithi143 ceph-mon[31461]: pgmap v513: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:47 smithi097 ceph-mon[28127]: pgmap v513: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:49 smithi111 ceph-mon[31001]: pgmap v514: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:50.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:49 smithi143 ceph-mon[31461]: pgmap v514: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:50.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:49 smithi097 ceph-mon[28127]: pgmap v514: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:51 smithi111 ceph-mon[31001]: pgmap v515: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:51 smithi143 ceph-mon[31461]: pgmap v515: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:51 smithi097 ceph-mon[28127]: pgmap v515: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:54.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:53 smithi111 ceph-mon[31001]: pgmap v516: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:53 smithi143 ceph-mon[31461]: pgmap v516: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:53 smithi097 ceph-mon[28127]: pgmap v516: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:35:56.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:55 smithi111 ceph-mon[31001]: pgmap v517: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:55 smithi143 ceph-mon[31461]: pgmap v517: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:56.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:55 smithi097 ceph-mon[28127]: pgmap v517: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:58.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:57 smithi143 ceph-mon[31461]: pgmap v518: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:58.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:57 smithi097 ceph-mon[28127]: pgmap v518: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:58.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:57 smithi111 ceph-mon[31001]: pgmap v518: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:35:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:35:58 smithi143 ceph-mon[31461]: pgmap v519: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:35:58 smithi097 ceph-mon[28127]: pgmap v519: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:35:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:35:58 smithi111 ceph-mon[31001]: pgmap v519: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:01 smithi111 ceph-mon[31001]: pgmap v520: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:02.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:01 smithi143 ceph-mon[31461]: pgmap v520: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:02.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:01 smithi097 ceph-mon[28127]: pgmap v520: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:03 smithi111 ceph-mon[31001]: pgmap v521: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:04.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:03 smithi143 ceph-mon[31461]: pgmap v521: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:04.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:03 smithi097 ceph-mon[28127]: pgmap v521: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:05 smithi111 ceph-mon[31001]: pgmap v522: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:06.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:05 smithi143 ceph-mon[31461]: pgmap v522: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:05 smithi097 ceph-mon[28127]: pgmap v522: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:08.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:07 smithi111 ceph-mon[31001]: pgmap v523: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:08.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:07 smithi143 ceph-mon[31461]: pgmap v523: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:08.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:07 smithi097 ceph-mon[28127]: pgmap v523: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:10.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:09 smithi111 ceph-mon[31001]: pgmap v524: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:09 smithi143 ceph-mon[31461]: pgmap v524: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:10.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:09 smithi097 ceph-mon[28127]: pgmap v524: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:12.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:11 smithi111 ceph-mon[31001]: pgmap v525: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:11 smithi143 ceph-mon[31461]: pgmap v525: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:12.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:11 smithi097 ceph-mon[28127]: pgmap v525: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:14.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:13 smithi111 ceph-mon[31001]: pgmap v526: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:13 smithi143 ceph-mon[31461]: pgmap v526: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:14.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:13 smithi097 ceph-mon[28127]: pgmap v526: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:16.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:15 smithi111 ceph-mon[31001]: pgmap v527: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:16.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:15 smithi143 ceph-mon[31461]: pgmap v527: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:16.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:15 smithi097 ceph-mon[28127]: pgmap v527: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:18.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:17 smithi143 ceph-mon[31461]: pgmap v528: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:18.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:17 smithi097 ceph-mon[28127]: pgmap v528: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:18.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:17 smithi111 ceph-mon[31001]: pgmap v528: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:36:18.655+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4a80034c0 con 0x7fa4d0104bc0 2024-04-26T21:36:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:36:18.655+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fa4a80034c0 con 0x7fa4d0104bc0 2024-04-26T21:36:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:19 smithi143 ceph-mon[31461]: pgmap v529: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:20.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:19 smithi097 ceph-mon[28127]: pgmap v529: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:20.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:19 smithi111 ceph-mon[31001]: pgmap v529: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:20 smithi143 ceph-mon[31461]: pgmap v530: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:21.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:20 smithi097 ceph-mon[28127]: pgmap v530: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:21.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:20 smithi111 ceph-mon[31001]: pgmap v530: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:23 smithi111 ceph-mon[31001]: pgmap v531: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:23 smithi143 ceph-mon[31461]: pgmap v531: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:24.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:23 smithi097 ceph-mon[28127]: pgmap v531: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:26.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:25 smithi111 ceph-mon[31001]: pgmap v532: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:25 smithi143 ceph-mon[31461]: pgmap v532: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:25 smithi097 ceph-mon[28127]: pgmap v532: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:28.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:27 smithi111 ceph-mon[31001]: pgmap v533: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:28.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:27 smithi143 ceph-mon[31461]: pgmap v533: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:27 smithi097 ceph-mon[28127]: pgmap v533: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:30.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:29 smithi111 ceph-mon[31001]: pgmap v534: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:29 smithi143 ceph-mon[31461]: pgmap v534: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:29 smithi097 ceph-mon[28127]: pgmap v534: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:32.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:31 smithi111 ceph-mon[31001]: pgmap v535: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:32.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:31 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:32.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:31 smithi143 ceph-mon[31461]: pgmap v535: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:32.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:31 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:32.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:31 smithi097 ceph-mon[28127]: pgmap v535: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:32.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:31 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:33.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:32 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:36:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:33.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:33.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:32 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:36:33.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:33.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:33.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:32 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:36:34.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:33 smithi111 ceph-mon[31001]: pgmap v536: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:34.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:33 smithi143 ceph-mon[31461]: pgmap v536: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:33 smithi097 ceph-mon[28127]: pgmap v536: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:36.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:35 smithi143 ceph-mon[31461]: pgmap v537: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:36.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:35 smithi097 ceph-mon[28127]: pgmap v537: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:36.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:35 smithi111 ceph-mon[31001]: pgmap v537: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:37 smithi143 ceph-mon[31461]: pgmap v538: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:38.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:37 smithi097 ceph-mon[28127]: pgmap v538: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:38.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:37 smithi111 ceph-mon[31001]: pgmap v538: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:39 smithi143 ceph-mon[31461]: pgmap v539: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:40.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:39 smithi097 ceph-mon[28127]: pgmap v539: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:40.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:39 smithi111 ceph-mon[31001]: pgmap v539: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:41.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:40 smithi111 ceph-mon[31001]: pgmap v540: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:41.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:40 smithi143 ceph-mon[31461]: pgmap v540: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:41.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:40 smithi097 ceph-mon[28127]: pgmap v540: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:44.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:43 smithi111 ceph-mon[31001]: pgmap v541: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:43 smithi143 ceph-mon[31461]: pgmap v541: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:44.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:43 smithi097 ceph-mon[28127]: pgmap v541: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:46.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:45 smithi111 ceph-mon[31001]: pgmap v542: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:45 smithi143 ceph-mon[31461]: pgmap v542: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:46.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:45 smithi097 ceph-mon[28127]: pgmap v542: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-26T21:36:48.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:47 smithi111 ceph-mon[31001]: pgmap v543: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:47 smithi143 ceph-mon[31461]: pgmap v543: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:47 smithi097 ceph-mon[28127]: pgmap v543: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:50.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:49 smithi111 ceph-mon[31001]: pgmap v544: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:50.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:49 smithi143 ceph-mon[31461]: pgmap v544: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:50.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:49 smithi097 ceph-mon[28127]: pgmap v544: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:52.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:51 smithi111 ceph-mon[31001]: pgmap v545: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:51 smithi143 ceph-mon[31461]: pgmap v545: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:51 smithi097 ceph-mon[28127]: pgmap v545: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:53 smithi143 ceph-mon[31461]: pgmap v546: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:53 smithi097 ceph-mon[28127]: pgmap v546: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:54.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:53 smithi111 ceph-mon[31001]: pgmap v546: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:36:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:55 smithi143 ceph-mon[31461]: pgmap v547: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:56.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:55 smithi097 ceph-mon[28127]: pgmap v547: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:56.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:55 smithi111 ceph-mon[31001]: pgmap v547: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:58.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:57 smithi143 ceph-mon[31461]: pgmap v548: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:58.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:57 smithi097 ceph-mon[28127]: pgmap v548: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:58.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:57 smithi111 ceph-mon[31001]: pgmap v548: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:36:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:36:58 smithi143 ceph-mon[31461]: pgmap v549: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:36:58 smithi097 ceph-mon[28127]: pgmap v549: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:36:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:36:58 smithi111 ceph-mon[31001]: pgmap v549: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:02.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:01 smithi111 ceph-mon[31001]: pgmap v550: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:02.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:01 smithi143 ceph-mon[31461]: pgmap v550: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:02.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:01 smithi097 ceph-mon[28127]: pgmap v550: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:04.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:03 smithi111 ceph-mon[31001]: pgmap v551: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:04.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:03 smithi143 ceph-mon[31461]: pgmap v551: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:04.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:03 smithi097 ceph-mon[28127]: pgmap v551: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:06.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:05 smithi111 ceph-mon[31001]: pgmap v552: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:06.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:05 smithi143 ceph-mon[31461]: pgmap v552: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:05 smithi097 ceph-mon[28127]: pgmap v552: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:08.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:07 smithi143 ceph-mon[31461]: pgmap v553: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:08.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:07 smithi097 ceph-mon[28127]: pgmap v553: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:08.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:07 smithi111 ceph-mon[31001]: pgmap v553: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:09 smithi143 ceph-mon[31461]: pgmap v554: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:10.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:09 smithi097 ceph-mon[28127]: pgmap v554: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:10.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:09 smithi111 ceph-mon[31001]: pgmap v554: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:12.195 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:11 smithi111 ceph-mon[31001]: pgmap v555: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:11 smithi143 ceph-mon[31461]: pgmap v555: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:12.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:11 smithi097 ceph-mon[28127]: pgmap v555: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:13 smithi143 ceph-mon[31461]: pgmap v556: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:14.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:13 smithi097 ceph-mon[28127]: pgmap v556: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:14.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:13 smithi111 ceph-mon[31001]: pgmap v556: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:16.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:15 smithi143 ceph-mon[31461]: pgmap v557: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:16.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:15 smithi097 ceph-mon[28127]: pgmap v557: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:16.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:15 smithi111 ceph-mon[31001]: pgmap v557: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:16 smithi143 ceph-mon[31461]: pgmap v558: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:16 smithi097 ceph-mon[28127]: pgmap v558: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:17.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:16 smithi111 ceph-mon[31001]: pgmap v558: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:19 smithi111 ceph-mon[31001]: pgmap v559: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:19 smithi143 ceph-mon[31461]: pgmap v559: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:20.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:19 smithi097 ceph-mon[28127]: pgmap v559: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:21 smithi111 ceph-mon[31001]: pgmap v560: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:21 smithi143 ceph-mon[31461]: pgmap v560: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:22.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:21 smithi097 ceph-mon[28127]: pgmap v560: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:23 smithi143 ceph-mon[31461]: pgmap v561: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:24.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:23 smithi097 ceph-mon[28127]: pgmap v561: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:24.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:23 smithi111 ceph-mon[31001]: pgmap v561: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:25 smithi143 ceph-mon[31461]: pgmap v562: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:25 smithi097 ceph-mon[28127]: pgmap v562: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:25 smithi111 ceph-mon[31001]: pgmap v562: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:28.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:27 smithi143 ceph-mon[31461]: pgmap v563: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:27 smithi097 ceph-mon[28127]: pgmap v563: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:27 smithi111 ceph-mon[31001]: pgmap v563: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:29 smithi143 ceph-mon[31461]: pgmap v564: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:29 smithi097 ceph-mon[28127]: pgmap v564: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:30.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:29 smithi111 ceph-mon[31001]: pgmap v564: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:32.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:31 smithi143 ceph-mon[31461]: pgmap v565: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:32.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:31 smithi097 ceph-mon[28127]: pgmap v565: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:31 smithi111 ceph-mon[31001]: pgmap v565: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:32 smithi143 ceph-mon[31461]: pgmap v566: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:33.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:32 smithi097 ceph-mon[28127]: pgmap v566: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:32 smithi111 ceph-mon[31001]: pgmap v566: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:34.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:33 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:34.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:33 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:33 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:35.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:35.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:37:35.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:34 smithi143 ceph-mon[31461]: pgmap v567: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:35.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:37:35.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:34 smithi097 ceph-mon[28127]: pgmap v567: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:37:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:34 smithi111 ceph-mon[31001]: pgmap v567: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:37 smithi111 ceph-mon[31001]: pgmap v568: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:37 smithi143 ceph-mon[31461]: pgmap v568: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:38.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:37 smithi097 ceph-mon[28127]: pgmap v568: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:39 smithi143 ceph-mon[31461]: pgmap v569: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:40.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:39 smithi097 ceph-mon[28127]: pgmap v569: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:40.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:39 smithi111 ceph-mon[31001]: pgmap v569: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:42.194 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:41 smithi111 ceph-mon[31001]: pgmap v570: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:41 smithi143 ceph-mon[31461]: pgmap v570: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:42.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:41 smithi097 ceph-mon[28127]: pgmap v570: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:43 smithi143 ceph-mon[31461]: pgmap v571: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:44.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:43 smithi097 ceph-mon[28127]: pgmap v571: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:44.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:43 smithi111 ceph-mon[31001]: pgmap v571: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:45 smithi143 ceph-mon[31461]: pgmap v572: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:46.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:45 smithi097 ceph-mon[28127]: pgmap v572: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:46.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:45 smithi111 ceph-mon[31001]: pgmap v572: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:47 smithi143 ceph-mon[31461]: pgmap v573: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:47 smithi097 ceph-mon[28127]: pgmap v573: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:48.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:47 smithi111 ceph-mon[31001]: pgmap v573: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:48 smithi143 ceph-mon[31461]: pgmap v574: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:48 smithi097 ceph-mon[28127]: pgmap v574: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:48 smithi111 ceph-mon[31001]: pgmap v574: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:51 smithi143 ceph-mon[31461]: pgmap v575: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:51 smithi097 ceph-mon[28127]: pgmap v575: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:52.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:51 smithi111 ceph-mon[31001]: pgmap v575: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:53 smithi143 ceph-mon[31461]: pgmap v576: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:53 smithi097 ceph-mon[28127]: pgmap v576: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:54.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:53 smithi111 ceph-mon[31001]: pgmap v576: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:37:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:55 smithi143 ceph-mon[31461]: pgmap v577: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:56.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:55 smithi097 ceph-mon[28127]: pgmap v577: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:56.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:55 smithi111 ceph-mon[31001]: pgmap v577: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:37:58.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:57 smithi143 ceph-mon[31461]: pgmap v578: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:58.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:57 smithi097 ceph-mon[28127]: pgmap v578: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:37:58.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:57 smithi111 ceph-mon[31001]: pgmap v578: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:00.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:37:59 smithi143 ceph-mon[31461]: pgmap v579: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:00.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:37:59 smithi097 ceph-mon[28127]: pgmap v579: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:37:59 smithi111 ceph-mon[31001]: pgmap v579: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:01.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:00 smithi143 ceph-mon[31461]: pgmap v580: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:00 smithi097 ceph-mon[28127]: pgmap v580: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:00 smithi111 ceph-mon[31001]: pgmap v580: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:04.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:03 smithi143 ceph-mon[31461]: pgmap v581: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:04.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:03 smithi097 ceph-mon[28127]: pgmap v581: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:04.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:03 smithi111 ceph-mon[31001]: pgmap v581: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:06.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:05 smithi143 ceph-mon[31461]: pgmap v582: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:05 smithi097 ceph-mon[28127]: pgmap v582: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:06.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:05 smithi111 ceph-mon[31001]: pgmap v582: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:08.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:07 smithi143 ceph-mon[31461]: pgmap v583: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:08.316 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:07 smithi097 ceph-mon[28127]: pgmap v583: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:08.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:07 smithi111 ceph-mon[31001]: pgmap v583: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:10.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:09 smithi143 ceph-mon[31461]: pgmap v584: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:10.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:09 smithi097 ceph-mon[28127]: pgmap v584: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:10.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:09 smithi111 ceph-mon[31001]: pgmap v584: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:12.195 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:11 smithi111 ceph-mon[31001]: pgmap v585: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:11 smithi143 ceph-mon[31461]: pgmap v585: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:12.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:11 smithi097 ceph-mon[28127]: pgmap v585: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:13.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:12 smithi143 ceph-mon[31461]: pgmap v586: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:12 smithi097 ceph-mon[28127]: pgmap v586: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:13.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:12 smithi111 ceph-mon[31001]: pgmap v586: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:16.297 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:15 smithi143 ceph-mon[31461]: pgmap v587: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:16.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:15 smithi097 ceph-mon[28127]: pgmap v587: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:16.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:15 smithi111 ceph-mon[31001]: pgmap v587: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:18.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:17 smithi143 ceph-mon[31461]: pgmap v588: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:18.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:17 smithi097 ceph-mon[28127]: pgmap v588: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:18.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:17 smithi111 ceph-mon[31001]: pgmap v588: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:19 smithi143 ceph-mon[31461]: pgmap v589: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:20.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:19 smithi097 ceph-mon[28127]: pgmap v589: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:20.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:19 smithi111 ceph-mon[31001]: pgmap v589: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:21 smithi143 ceph-mon[31461]: pgmap v590: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:21 smithi097 ceph-mon[28127]: pgmap v590: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:22.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:21 smithi111 ceph-mon[31001]: pgmap v590: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:22 smithi143 ceph-mon[31461]: pgmap v591: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:22 smithi097 ceph-mon[28127]: pgmap v591: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:22 smithi111 ceph-mon[31001]: pgmap v591: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:25 smithi143 ceph-mon[31461]: pgmap v592: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:25 smithi097 ceph-mon[28127]: pgmap v592: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:25 smithi111 ceph-mon[31001]: pgmap v592: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:28.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:27 smithi143 ceph-mon[31461]: pgmap v593: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:27 smithi097 ceph-mon[28127]: pgmap v593: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:27 smithi111 ceph-mon[31001]: pgmap v593: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:29 smithi143 ceph-mon[31461]: pgmap v594: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:29 smithi097 ceph-mon[28127]: pgmap v594: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:30.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:29 smithi111 ceph-mon[31001]: pgmap v594: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:32.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:31 smithi143 ceph-mon[31461]: pgmap v595: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:32.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:31 smithi097 ceph-mon[28127]: pgmap v595: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:31 smithi111 ceph-mon[31001]: pgmap v595: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:32 smithi143 ceph-mon[31461]: pgmap v596: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:33.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:32 smithi097 ceph-mon[28127]: pgmap v596: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:32 smithi111 ceph-mon[31001]: pgmap v596: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:36.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:35 smithi143 ceph-mon[31461]: pgmap v597: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:36.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:35 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:36.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:35 smithi097 ceph-mon[28127]: pgmap v597: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:36.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:35 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:36.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:35 smithi111 ceph-mon[31001]: pgmap v597: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:36.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:35 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:37.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:38:37.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:38:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:38:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:37 smithi143 ceph-mon[31461]: pgmap v598: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:38.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:37 smithi097 ceph-mon[28127]: pgmap v598: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:38.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:37 smithi111 ceph-mon[31001]: pgmap v598: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:39 smithi143 ceph-mon[31461]: pgmap v599: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:40.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:39 smithi097 ceph-mon[28127]: pgmap v599: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:40.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:39 smithi111 ceph-mon[31001]: pgmap v599: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:41.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:40 smithi143 ceph-mon[31461]: pgmap v600: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:41.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:40 smithi097 ceph-mon[28127]: pgmap v600: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:41.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:40 smithi111 ceph-mon[31001]: pgmap v600: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:44.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:43 smithi143 ceph-mon[31461]: pgmap v601: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:44.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:43 smithi097 ceph-mon[28127]: pgmap v601: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:44.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:43 smithi111 ceph-mon[31001]: pgmap v601: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:46.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:45 smithi143 ceph-mon[31461]: pgmap v602: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:46.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:45 smithi097 ceph-mon[28127]: pgmap v602: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:46.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:45 smithi111 ceph-mon[31001]: pgmap v602: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:48.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:47 smithi143 ceph-mon[31461]: pgmap v603: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:48.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:47 smithi097 ceph-mon[28127]: pgmap v603: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:48.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:47 smithi111 ceph-mon[31001]: pgmap v603: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:48 smithi143 ceph-mon[31461]: pgmap v604: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:48 smithi097 ceph-mon[28127]: pgmap v604: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:48 smithi111 ceph-mon[31001]: pgmap v604: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:52.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:51 smithi143 ceph-mon[31461]: pgmap v605: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:52.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:51 smithi097 ceph-mon[28127]: pgmap v605: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:52.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:51 smithi111 ceph-mon[31001]: pgmap v605: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:54.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:53 smithi143 ceph-mon[31461]: pgmap v606: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:54.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:53 smithi097 ceph-mon[28127]: pgmap v606: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:54.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:53 smithi111 ceph-mon[31001]: pgmap v606: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:38:56.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:55 smithi143 ceph-mon[31461]: pgmap v607: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:56.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:55 smithi097 ceph-mon[28127]: pgmap v607: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:56.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:55 smithi111 ceph-mon[31001]: pgmap v607: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:38:57.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:56 smithi143 ceph-mon[31461]: pgmap v608: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:57.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:56 smithi097 ceph-mon[28127]: pgmap v608: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:38:57.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:56 smithi111 ceph-mon[31001]: pgmap v608: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:00.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:38:59 smithi143 ceph-mon[31461]: pgmap v609: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:00.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:38:59 smithi097 ceph-mon[28127]: pgmap v609: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:38:59 smithi111 ceph-mon[31001]: pgmap v609: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:02.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:01 smithi143 ceph-mon[31461]: pgmap v610: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:02.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:01 smithi097 ceph-mon[28127]: pgmap v610: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:02.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:01 smithi111 ceph-mon[31001]: pgmap v610: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:02 smithi143 ceph-mon[31461]: pgmap v611: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:03.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:02 smithi097 ceph-mon[28127]: pgmap v611: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:03.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:02 smithi111 ceph-mon[31001]: pgmap v611: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:06.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:05 smithi143 ceph-mon[31461]: pgmap v612: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:06.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:05 smithi097 ceph-mon[28127]: pgmap v612: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:06.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:05 smithi111 ceph-mon[31001]: pgmap v612: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:08.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:07 smithi143 ceph-mon[31461]: pgmap v613: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:08.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:07 smithi097 ceph-mon[28127]: pgmap v613: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:08.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:07 smithi111 ceph-mon[31001]: pgmap v613: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:08 smithi143 ceph-mon[31461]: pgmap v614: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:08 smithi097 ceph-mon[28127]: pgmap v614: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:09.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:08 smithi111 ceph-mon[31001]: pgmap v614: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:12.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:11 smithi143 ceph-mon[31461]: pgmap v615: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:12.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:11 smithi097 ceph-mon[28127]: pgmap v615: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:12.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:11 smithi111 ceph-mon[31001]: pgmap v615: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:14.195 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:13 smithi111 ceph-mon[31001]: pgmap v616: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:14.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:13 smithi143 ceph-mon[31461]: pgmap v616: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:14.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:13 smithi097 ceph-mon[28127]: pgmap v616: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:14 smithi143 ceph-mon[31461]: pgmap v617: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:14 smithi097 ceph-mon[28127]: pgmap v617: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:15.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:14 smithi111 ceph-mon[31001]: pgmap v617: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:18.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:17 smithi143 ceph-mon[31461]: pgmap v618: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:18.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:17 smithi097 ceph-mon[28127]: pgmap v618: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:18.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:17 smithi111 ceph-mon[31001]: pgmap v618: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:18.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:39:18.684+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4a8005850 con 0x7fa4d0104bc0 2024-04-26T21:39:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:39:18.684+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fa4a8005850 con 0x7fa4d0104bc0 2024-04-26T21:39:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:18 smithi143 ceph-mon[31461]: pgmap v619: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:18 smithi097 ceph-mon[28127]: pgmap v619: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:19.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:18 smithi111 ceph-mon[31001]: pgmap v619: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:21 smithi143 ceph-mon[31461]: pgmap v620: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:21 smithi097 ceph-mon[28127]: pgmap v620: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:22.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:21 smithi111 ceph-mon[31001]: pgmap v620: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:22 smithi143 ceph-mon[31461]: pgmap v621: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:22 smithi097 ceph-mon[28127]: pgmap v621: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:22 smithi111 ceph-mon[31001]: pgmap v621: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:26.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:25 smithi143 ceph-mon[31461]: pgmap v622: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:26.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:25 smithi097 ceph-mon[28127]: pgmap v622: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:25 smithi111 ceph-mon[31001]: pgmap v622: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:27.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:26 smithi143 ceph-mon[31461]: pgmap v623: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:27.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:26 smithi097 ceph-mon[28127]: pgmap v623: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:26 smithi111 ceph-mon[31001]: pgmap v623: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:30.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:29 smithi143 ceph-mon[31461]: pgmap v624: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:30.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:29 smithi097 ceph-mon[28127]: pgmap v624: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:30.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:29 smithi111 ceph-mon[31001]: pgmap v624: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:39:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:30 smithi143 ceph-mon[31461]: pgmap v625: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:39:31.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:30 smithi097 ceph-mon[28127]: pgmap v625: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:39:31.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:30 smithi111 ceph-mon[31001]: pgmap v625: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:39:34.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:33 smithi143 ceph-mon[31461]: pgmap v626: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:34.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:33 smithi097 ceph-mon[28127]: pgmap v626: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:33 smithi111 ceph-mon[31001]: pgmap v626: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:39:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:34 smithi143 ceph-mon[31461]: pgmap v627: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:35.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:34 smithi097 ceph-mon[28127]: pgmap v627: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:34 smithi111 ceph-mon[31001]: pgmap v627: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:37.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:36 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:36 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:37.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:36 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:38.067 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:37 smithi097 ceph-mon[28127]: pgmap v628: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:37 smithi111 ceph-mon[31001]: pgmap v628: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:38.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:37 smithi143 ceph-mon[31461]: pgmap v628: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:39.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:39.065 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:39:39.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:39.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:39:39.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:39.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:39:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:39 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-26T21:39:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:39 smithi097 ceph-mon[28127]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-26T21:39:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:39 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi097 to 176.4M 2024-04-26T21:39:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:39 smithi097 ceph-mon[28127]: Unable to set osd_memory_target on smithi097 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:39:40.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:39 smithi097 ceph-mon[28127]: pgmap v629: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:39 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-26T21:39:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:39 smithi111 ceph-mon[31001]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-26T21:39:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:39 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi097 to 176.4M 2024-04-26T21:39:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:39 smithi111 ceph-mon[31001]: Unable to set osd_memory_target on smithi097 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:39:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:39 smithi111 ceph-mon[31001]: pgmap v629: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:39 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-04-26T21:39:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:39 smithi143 ceph-mon[31461]: Unable to set osd_memory_target on smithi111 to 185063424: error parsing value: Value '185063424' is below minimum 939524096 2024-04-26T21:39:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:39 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi097 to 176.4M 2024-04-26T21:39:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:39 smithi143 ceph-mon[31461]: Unable to set osd_memory_target on smithi097 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:39:40.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:39 smithi143 ceph-mon[31461]: pgmap v629: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:42.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:41 smithi143 ceph-mon[31461]: pgmap v630: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:42.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:41 smithi097 ceph-mon[28127]: pgmap v630: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:42.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:41 smithi111 ceph-mon[31001]: pgmap v630: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:43.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:42 smithi143 ceph-mon[31461]: pgmap v631: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:43.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:42 smithi097 ceph-mon[28127]: pgmap v631: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:43.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:42 smithi111 ceph-mon[31001]: pgmap v631: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:39:45.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:44 smithi143 ceph-mon[31461]: pgmap v632: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:45.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:44 smithi097 ceph-mon[28127]: pgmap v632: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:45.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:44 smithi111 ceph-mon[31001]: pgmap v632: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:47.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:47 smithi143 ceph-mon[31461]: pgmap v633: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:47.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:47 smithi097 ceph-mon[28127]: pgmap v633: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:47 smithi111 ceph-mon[31001]: pgmap v633: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:49 smithi143 ceph-mon[31461]: pgmap v634: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:39:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:49 smithi097 ceph-mon[28127]: pgmap v634: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:39:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:49 smithi111 ceph-mon[31001]: pgmap v634: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:39:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:51 smithi143 ceph-mon[31461]: pgmap v635: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:51.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:51 smithi097 ceph-mon[28127]: pgmap v635: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:51.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:51 smithi111 ceph-mon[31001]: pgmap v635: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:53.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:53 smithi143 ceph-mon[31461]: pgmap v636: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:39:53.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:53 smithi097 ceph-mon[28127]: pgmap v636: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:39:53.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:53 smithi111 ceph-mon[31001]: pgmap v636: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:39:55.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:55 smithi143 ceph-mon[31461]: pgmap v637: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:55.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:55 smithi097 ceph-mon[28127]: pgmap v637: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:55.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:55 smithi111 ceph-mon[31001]: pgmap v637: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:39:57.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:57 smithi143 ceph-mon[31461]: pgmap v638: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:57.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:57 smithi097 ceph-mon[28127]: pgmap v638: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:57.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:57 smithi111 ceph-mon[31001]: pgmap v638: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:39:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:39:59 smithi143 ceph-mon[31461]: pgmap v639: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:39:59 smithi097 ceph-mon[28127]: pgmap v639: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:39:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:39:59 smithi111 ceph-mon[31001]: pgmap v639: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:00.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:00 smithi143 ceph-mon[31461]: overall HEALTH_OK 2024-04-26T21:40:00.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:00 smithi097 ceph-mon[28127]: overall HEALTH_OK 2024-04-26T21:40:00.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:00 smithi111 ceph-mon[31001]: overall HEALTH_OK 2024-04-26T21:40:01.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:01 smithi143 ceph-mon[31461]: pgmap v640: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:01 smithi097 ceph-mon[28127]: pgmap v640: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:01 smithi111 ceph-mon[31001]: pgmap v640: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:03 smithi143 ceph-mon[31461]: pgmap v641: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:40:03.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:03 smithi097 ceph-mon[28127]: pgmap v641: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:40:03.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:03 smithi111 ceph-mon[31001]: pgmap v641: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:40:05.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:05 smithi143 ceph-mon[31461]: pgmap v642: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:05 smithi097 ceph-mon[28127]: pgmap v642: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:05.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:05 smithi111 ceph-mon[31001]: pgmap v642: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:07.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:07 smithi143 ceph-mon[31461]: pgmap v643: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:40:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:07 smithi097 ceph-mon[28127]: pgmap v643: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:40:07.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:07 smithi111 ceph-mon[31001]: pgmap v643: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:40:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:09 smithi143 ceph-mon[31461]: pgmap v644: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:09 smithi097 ceph-mon[28127]: pgmap v644: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:09.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:09 smithi111 ceph-mon[31001]: pgmap v644: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:11.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:11 smithi143 ceph-mon[31461]: pgmap v645: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:11.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:11 smithi097 ceph-mon[28127]: pgmap v645: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:11 smithi111 ceph-mon[31001]: pgmap v645: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:13.297 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:13 smithi143 ceph-mon[31461]: pgmap v646: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:13 smithi097 ceph-mon[28127]: pgmap v646: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:13.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:13 smithi111 ceph-mon[31001]: pgmap v646: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:15 smithi143 ceph-mon[31461]: pgmap v647: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:15 smithi097 ceph-mon[28127]: pgmap v647: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:15.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:15 smithi111 ceph-mon[31001]: pgmap v647: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:17 smithi143 ceph-mon[31461]: pgmap v648: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:17 smithi097 ceph-mon[28127]: pgmap v648: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:17.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:17 smithi111 ceph-mon[31001]: pgmap v648: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:19 smithi143 ceph-mon[31461]: pgmap v649: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:19 smithi097 ceph-mon[28127]: pgmap v649: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:19.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:19 smithi111 ceph-mon[31001]: pgmap v649: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:21 smithi143 ceph-mon[31461]: pgmap v650: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:21.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:21 smithi097 ceph-mon[28127]: pgmap v650: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:21.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:21 smithi111 ceph-mon[31001]: pgmap v650: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:23 smithi143 ceph-mon[31461]: pgmap v651: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:23 smithi097 ceph-mon[28127]: pgmap v651: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:23 smithi111 ceph-mon[31001]: pgmap v651: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:25.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:25 smithi143 ceph-mon[31461]: pgmap v652: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:25.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:25 smithi097 ceph-mon[28127]: pgmap v652: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:25.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:25 smithi111 ceph-mon[31001]: pgmap v652: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:27.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:27 smithi143 ceph-mon[31461]: pgmap v653: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:27.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:27 smithi097 ceph-mon[28127]: pgmap v653: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:27 smithi111 ceph-mon[31001]: pgmap v653: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:29.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:29 smithi143 ceph-mon[31461]: pgmap v654: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:29.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:29 smithi097 ceph-mon[28127]: pgmap v654: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:29.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:29 smithi111 ceph-mon[31001]: pgmap v654: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:31 smithi143 ceph-mon[31461]: pgmap v655: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:31.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:31 smithi097 ceph-mon[28127]: pgmap v655: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:31.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:31 smithi111 ceph-mon[31001]: pgmap v655: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:33 smithi143 ceph-mon[31461]: pgmap v656: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:33.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:33 smithi097 ceph-mon[28127]: pgmap v656: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:33 smithi111 ceph-mon[31001]: pgmap v656: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:35 smithi143 ceph-mon[31461]: pgmap v657: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:35.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:35 smithi097 ceph-mon[28127]: pgmap v657: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:35.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:35 smithi111 ceph-mon[31001]: pgmap v657: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:37.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:37 smithi143 ceph-mon[31461]: pgmap v658: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:37.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:37 smithi097 ceph-mon[28127]: pgmap v658: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:37 smithi111 ceph-mon[31001]: pgmap v658: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:39.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:38 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:39.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:38 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:39.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:38 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:40.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:39 smithi143 ceph-mon[31461]: pgmap v659: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:40.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:39 smithi097 ceph-mon[28127]: pgmap v659: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:40.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:39 smithi111 ceph-mon[31001]: pgmap v659: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:41.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:41.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:40:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:40:41.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:41.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:41.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:40:42.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:41 smithi143 ceph-mon[31461]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-04-26T21:40:42.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:41 smithi143 ceph-mon[31461]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:40:42.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:41 smithi143 ceph-mon[31461]: pgmap v660: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:42.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:41 smithi097 ceph-mon[28127]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-04-26T21:40:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:41 smithi097 ceph-mon[28127]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:40:42.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:41 smithi097 ceph-mon[28127]: pgmap v660: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:41 smithi111 ceph-mon[31001]: Adjusting osd_memory_target on smithi143 to 176.4M 2024-04-26T21:40:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:41 smithi111 ceph-mon[31001]: Unable to set osd_memory_target on smithi143 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-26T21:40:42.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:41 smithi111 ceph-mon[31001]: pgmap v660: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:43.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:43 smithi143 ceph-mon[31461]: pgmap v661: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:43.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:43 smithi097 ceph-mon[28127]: pgmap v661: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:43.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:43 smithi111 ceph-mon[31001]: pgmap v661: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:45.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:45 smithi143 ceph-mon[31461]: pgmap v662: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:45.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:45 smithi097 ceph-mon[28127]: pgmap v662: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:45.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:45 smithi111 ceph-mon[31001]: pgmap v662: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:47.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:47 smithi143 ceph-mon[31461]: pgmap v663: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:47.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:47 smithi097 ceph-mon[28127]: pgmap v663: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:47 smithi111 ceph-mon[31001]: pgmap v663: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:49 smithi143 ceph-mon[31461]: pgmap v664: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:49 smithi097 ceph-mon[28127]: pgmap v664: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:49 smithi111 ceph-mon[31001]: pgmap v664: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:51 smithi143 ceph-mon[31461]: pgmap v665: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:51 smithi097 ceph-mon[28127]: pgmap v665: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:51.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:51 smithi111 ceph-mon[31001]: pgmap v665: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:53.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:53 smithi143 ceph-mon[31461]: pgmap v666: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:53.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:53 smithi097 ceph-mon[28127]: pgmap v666: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:53.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:53 smithi111 ceph-mon[31001]: pgmap v666: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:40:55.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:55 smithi143 ceph-mon[31461]: pgmap v667: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:55.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:55 smithi097 ceph-mon[28127]: pgmap v667: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:55.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:55 smithi111 ceph-mon[31001]: pgmap v667: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:40:57.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:57 smithi143 ceph-mon[31461]: pgmap v668: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:57.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:57 smithi097 ceph-mon[28127]: pgmap v668: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:57.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:57 smithi111 ceph-mon[31001]: pgmap v668: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:40:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:40:59 smithi143 ceph-mon[31461]: pgmap v669: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:40:59 smithi097 ceph-mon[28127]: pgmap v669: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:40:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:40:59 smithi111 ceph-mon[31001]: pgmap v669: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:01.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:01 smithi143 ceph-mon[31461]: pgmap v670: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:01 smithi097 ceph-mon[28127]: pgmap v670: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:01 smithi111 ceph-mon[31001]: pgmap v670: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:03 smithi143 ceph-mon[31461]: pgmap v671: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:41:03.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:03 smithi097 ceph-mon[28127]: pgmap v671: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:41:03.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:03 smithi111 ceph-mon[31001]: pgmap v671: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:41:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:05 smithi143 ceph-mon[31461]: pgmap v672: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:05 smithi097 ceph-mon[28127]: pgmap v672: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:05.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:05 smithi111 ceph-mon[31001]: pgmap v672: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:07.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:07 smithi143 ceph-mon[31461]: pgmap v673: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:41:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:07 smithi097 ceph-mon[28127]: pgmap v673: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:41:07.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:07 smithi111 ceph-mon[31001]: pgmap v673: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:41:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:09 smithi143 ceph-mon[31461]: pgmap v674: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:09 smithi097 ceph-mon[28127]: pgmap v674: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:09.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:09 smithi111 ceph-mon[31001]: pgmap v674: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:41:11.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:11 smithi143 ceph-mon[31461]: pgmap v675: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:11.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:11 smithi097 ceph-mon[28127]: pgmap v675: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:11 smithi111 ceph-mon[31001]: pgmap v675: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:13.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:13 smithi143 ceph-mon[31461]: pgmap v676: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:13 smithi097 ceph-mon[28127]: pgmap v676: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:13.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:13 smithi111 ceph-mon[31001]: pgmap v676: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:15 smithi143 ceph-mon[31461]: pgmap v677: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:15 smithi097 ceph-mon[28127]: pgmap v677: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:15.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:15 smithi111 ceph-mon[31001]: pgmap v677: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:17 smithi143 ceph-mon[31461]: pgmap v678: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:17 smithi097 ceph-mon[28127]: pgmap v678: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:17.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:17 smithi111 ceph-mon[31001]: pgmap v678: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:19 smithi143 ceph-mon[31461]: pgmap v679: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:19 smithi097 ceph-mon[28127]: pgmap v679: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:19.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:19 smithi111 ceph-mon[31001]: pgmap v679: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:21 smithi143 ceph-mon[31461]: pgmap v680: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:21.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:21 smithi097 ceph-mon[28127]: pgmap v680: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:21.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:21 smithi111 ceph-mon[31001]: pgmap v680: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:23 smithi143 ceph-mon[31461]: pgmap v681: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:23 smithi097 ceph-mon[28127]: pgmap v681: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:23.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:23 smithi111 ceph-mon[31001]: pgmap v681: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:25.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:25 smithi143 ceph-mon[31461]: pgmap v682: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:25.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:25 smithi097 ceph-mon[28127]: pgmap v682: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:25.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:25 smithi111 ceph-mon[31001]: pgmap v682: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:27.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:27 smithi143 ceph-mon[31461]: pgmap v683: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:27.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:27 smithi097 ceph-mon[28127]: pgmap v683: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:27 smithi111 ceph-mon[31001]: pgmap v683: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:29.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:29 smithi143 ceph-mon[31461]: pgmap v684: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:29.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:29 smithi097 ceph-mon[28127]: pgmap v684: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:29.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:29 smithi111 ceph-mon[31001]: pgmap v684: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:31 smithi143 ceph-mon[31461]: pgmap v685: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:31.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:31 smithi097 ceph-mon[28127]: pgmap v685: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:31.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:31 smithi111 ceph-mon[31001]: pgmap v685: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:33 smithi143 ceph-mon[31461]: pgmap v686: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:33.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:33 smithi097 ceph-mon[28127]: pgmap v686: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:33 smithi111 ceph-mon[31001]: pgmap v686: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:33.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:41:33.705+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fa4a8005b90 con 0x7fa4d0104bc0 2024-04-26T21:41:33.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:41:33.706+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fa4a8005b90 con 0x7fa4d0104bc0 2024-04-26T21:41:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:35 smithi143 ceph-mon[31461]: pgmap v687: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:35.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:35 smithi097 ceph-mon[28127]: pgmap v687: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:35 smithi111 ceph-mon[31001]: pgmap v687: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:37.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:37 smithi143 ceph-mon[31461]: pgmap v688: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:37.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:37 smithi097 ceph-mon[28127]: pgmap v688: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:37 smithi111 ceph-mon[31001]: pgmap v688: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:39.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:39 smithi143 ceph-mon[31461]: pgmap v689: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:39.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:39 smithi097 ceph-mon[28127]: pgmap v689: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:39.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:39 smithi111 ceph-mon[31001]: pgmap v689: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:40.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:40 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:40 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:41.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:40 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:41.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:41 smithi111 ceph-mon[31001]: pgmap v690: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:42.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:41 smithi143 ceph-mon[31461]: pgmap v690: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:42.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:41 smithi097 ceph-mon[28127]: pgmap v690: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:41:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:41:43.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:43.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:43.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:41:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:43 smithi111 ceph-mon[31001]: pgmap v691: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:44.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:43 smithi143 ceph-mon[31461]: pgmap v691: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:44.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:43 smithi097 ceph-mon[28127]: pgmap v691: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:45.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:45 smithi143 ceph-mon[31461]: pgmap v692: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:45.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:45 smithi097 ceph-mon[28127]: pgmap v692: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:45.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:45 smithi111 ceph-mon[31001]: pgmap v692: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:47.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:47 smithi143 ceph-mon[31461]: pgmap v693: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:47.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:47 smithi097 ceph-mon[28127]: pgmap v693: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:47 smithi111 ceph-mon[31001]: pgmap v693: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:49 smithi143 ceph-mon[31461]: pgmap v694: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:49 smithi097 ceph-mon[28127]: pgmap v694: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:49 smithi111 ceph-mon[31001]: pgmap v694: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:51 smithi143 ceph-mon[31461]: pgmap v695: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:51 smithi097 ceph-mon[28127]: pgmap v695: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:51.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:51 smithi111 ceph-mon[31001]: pgmap v695: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:53.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:53 smithi143 ceph-mon[31461]: pgmap v696: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:53.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:53 smithi097 ceph-mon[28127]: pgmap v696: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:53.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:53 smithi111 ceph-mon[31001]: pgmap v696: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:41:55.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:55 smithi143 ceph-mon[31461]: pgmap v697: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:55.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:55 smithi097 ceph-mon[28127]: pgmap v697: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:55.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:55 smithi111 ceph-mon[31001]: pgmap v697: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:57.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:57 smithi143 ceph-mon[31461]: pgmap v698: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:57.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:57 smithi097 ceph-mon[28127]: pgmap v698: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:57.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:57 smithi111 ceph-mon[31001]: pgmap v698: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:41:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:41:59 smithi143 ceph-mon[31461]: pgmap v699: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:41:59 smithi097 ceph-mon[28127]: pgmap v699: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:41:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:41:59 smithi111 ceph-mon[31001]: pgmap v699: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:01.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:01 smithi143 ceph-mon[31461]: pgmap v700: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:01 smithi097 ceph-mon[28127]: pgmap v700: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:01.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:01 smithi111 ceph-mon[31001]: pgmap v700: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:03 smithi143 ceph-mon[31461]: pgmap v701: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:03.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:03 smithi097 ceph-mon[28127]: pgmap v701: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:03.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:03 smithi111 ceph-mon[31001]: pgmap v701: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:05 smithi143 ceph-mon[31461]: pgmap v702: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:05 smithi097 ceph-mon[28127]: pgmap v702: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:05.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:05 smithi111 ceph-mon[31001]: pgmap v702: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:07.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:07 smithi143 ceph-mon[31461]: pgmap v703: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:07 smithi097 ceph-mon[28127]: pgmap v703: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:07.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:07 smithi111 ceph-mon[31001]: pgmap v703: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:09 smithi143 ceph-mon[31461]: pgmap v704: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:09 smithi097 ceph-mon[28127]: pgmap v704: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:09.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:09 smithi111 ceph-mon[31001]: pgmap v704: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:11.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:11 smithi143 ceph-mon[31461]: pgmap v705: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:11.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:11 smithi097 ceph-mon[28127]: pgmap v705: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:11 smithi111 ceph-mon[31001]: pgmap v705: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:13.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:13 smithi143 ceph-mon[31461]: pgmap v706: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:13 smithi097 ceph-mon[28127]: pgmap v706: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:13.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:13 smithi111 ceph-mon[31001]: pgmap v706: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:15 smithi143 ceph-mon[31461]: pgmap v707: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:15 smithi097 ceph-mon[28127]: pgmap v707: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:15.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:15 smithi111 ceph-mon[31001]: pgmap v707: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:17 smithi143 ceph-mon[31461]: pgmap v708: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:17 smithi097 ceph-mon[28127]: pgmap v708: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:17.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:17 smithi111 ceph-mon[31001]: pgmap v708: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:42:18.712+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4a8005ef0 con 0x7fa4d0104bc0 2024-04-26T21:42:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:42:18.713+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fa4a8005ef0 con 0x7fa4d0104bc0 2024-04-26T21:42:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:19 smithi143 ceph-mon[31461]: pgmap v709: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:19 smithi097 ceph-mon[28127]: pgmap v709: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:19.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:19 smithi111 ceph-mon[31001]: pgmap v709: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:21 smithi143 ceph-mon[31461]: pgmap v710: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:21.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:21 smithi097 ceph-mon[28127]: pgmap v710: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:21.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:21 smithi111 ceph-mon[31001]: pgmap v710: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:23 smithi143 ceph-mon[31461]: pgmap v711: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:23 smithi097 ceph-mon[28127]: pgmap v711: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:23.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:23 smithi111 ceph-mon[31001]: pgmap v711: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:25.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:25 smithi143 ceph-mon[31461]: pgmap v712: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:25.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:25 smithi097 ceph-mon[28127]: pgmap v712: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:25.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:25 smithi111 ceph-mon[31001]: pgmap v712: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:27.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:27 smithi143 ceph-mon[31461]: pgmap v713: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:27.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:27 smithi097 ceph-mon[28127]: pgmap v713: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:27 smithi111 ceph-mon[31001]: pgmap v713: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:29.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:29 smithi143 ceph-mon[31461]: pgmap v714: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:29.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:29 smithi097 ceph-mon[28127]: pgmap v714: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:29.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:29 smithi111 ceph-mon[31001]: pgmap v714: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:31 smithi143 ceph-mon[31461]: pgmap v715: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:31.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:31 smithi097 ceph-mon[28127]: pgmap v715: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:31.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:31 smithi111 ceph-mon[31001]: pgmap v715: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:33.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:33 smithi143 ceph-mon[31461]: pgmap v716: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:33.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:33 smithi097 ceph-mon[28127]: pgmap v716: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:33 smithi111 ceph-mon[31001]: pgmap v716: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:35.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:35 smithi143 ceph-mon[31461]: pgmap v717: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:35.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:35 smithi097 ceph-mon[28127]: pgmap v717: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:35.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:35 smithi111 ceph-mon[31001]: pgmap v717: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:37.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:37 smithi143 ceph-mon[31461]: pgmap v718: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:37.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:37 smithi097 ceph-mon[28127]: pgmap v718: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:37.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:37 smithi111 ceph-mon[31001]: pgmap v718: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:39.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:39 smithi143 ceph-mon[31461]: pgmap v719: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:39.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:39 smithi097 ceph-mon[28127]: pgmap v719: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:39.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:39 smithi111 ceph-mon[31001]: pgmap v719: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:41.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:41 smithi143 ceph-mon[31461]: pgmap v720: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:41.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:41 smithi097 ceph-mon[28127]: pgmap v720: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:41.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:41 smithi111 ceph-mon[31001]: pgmap v720: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:42.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:42 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:42.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:42 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:42.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:42 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:43.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:43 smithi143 ceph-mon[31461]: pgmap v721: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:43.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:43 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:43 smithi097 ceph-mon[28127]: pgmap v721: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:43.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:43 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:43 smithi111 ceph-mon[31001]: pgmap v721: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:43.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:43.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:43 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:44.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:44.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:44.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:44 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:45.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:45.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:45.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:44 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:45.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:45.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:45.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:44 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:42:45.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:45 smithi111 ceph-mon[31001]: pgmap v722: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:46.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:45 smithi143 ceph-mon[31461]: pgmap v722: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:46.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:45 smithi097 ceph-mon[28127]: pgmap v722: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:47.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:47 smithi143 ceph-mon[31461]: pgmap v723: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:47.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:47 smithi097 ceph-mon[28127]: pgmap v723: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:47 smithi111 ceph-mon[31001]: pgmap v723: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:49 smithi143 ceph-mon[31461]: pgmap v724: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:49 smithi097 ceph-mon[28127]: pgmap v724: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:49.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:49 smithi111 ceph-mon[31001]: pgmap v724: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:51 smithi143 ceph-mon[31461]: pgmap v725: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:51 smithi097 ceph-mon[28127]: pgmap v725: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:51.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:51 smithi111 ceph-mon[31001]: pgmap v725: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:53.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:53 smithi097 ceph-mon[28127]: pgmap v726: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:53.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:53 smithi111 ceph-mon[31001]: pgmap v726: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:53.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:53 smithi143 ceph-mon[31461]: pgmap v726: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:42:55.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:55 smithi097 ceph-mon[28127]: pgmap v727: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:55.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:55 smithi111 ceph-mon[31001]: pgmap v727: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:55.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:55 smithi143 ceph-mon[31461]: pgmap v727: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:57.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:57 smithi097 ceph-mon[28127]: pgmap v728: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:57.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:57 smithi111 ceph-mon[31001]: pgmap v728: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:57.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:57 smithi143 ceph-mon[31461]: pgmap v728: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:42:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:42:59 smithi097 ceph-mon[28127]: pgmap v729: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:59.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:42:59 smithi111 ceph-mon[31001]: pgmap v729: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:42:59.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:42:59 smithi143 ceph-mon[31461]: pgmap v729: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:01 smithi097 ceph-mon[28127]: pgmap v730: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:01.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:01 smithi111 ceph-mon[31001]: pgmap v730: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:01.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:01 smithi143 ceph-mon[31461]: pgmap v730: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:03.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:03 smithi097 ceph-mon[28127]: pgmap v731: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:03.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:03 smithi111 ceph-mon[31001]: pgmap v731: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:03.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:03 smithi143 ceph-mon[31461]: pgmap v731: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:05 smithi097 ceph-mon[28127]: pgmap v732: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:05.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:05 smithi111 ceph-mon[31001]: pgmap v732: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:05.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:05 smithi143 ceph-mon[31461]: pgmap v732: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:07 smithi097 ceph-mon[28127]: pgmap v733: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:07.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:07 smithi111 ceph-mon[31001]: pgmap v733: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:07.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:07 smithi143 ceph-mon[31461]: pgmap v733: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:10.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:10 smithi097 ceph-mon[28127]: pgmap v734: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:10.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:10 smithi111 ceph-mon[31001]: pgmap v734: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:10.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:10 smithi143 ceph-mon[31461]: pgmap v734: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:12.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:12 smithi097 ceph-mon[28127]: pgmap v735: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:12.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:12 smithi111 ceph-mon[31001]: pgmap v735: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:12.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:12 smithi143 ceph-mon[31461]: pgmap v735: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:14.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:14 smithi097 ceph-mon[28127]: pgmap v736: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:14.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:14 smithi111 ceph-mon[31001]: pgmap v736: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:14.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:14 smithi143 ceph-mon[31461]: pgmap v736: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:16.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:16 smithi111 ceph-mon[31001]: pgmap v737: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:16.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:16 smithi143 ceph-mon[31461]: pgmap v737: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:16.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:16 smithi097 ceph-mon[28127]: pgmap v737: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:18.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:18 smithi111 ceph-mon[31001]: pgmap v738: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:18.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:18 smithi143 ceph-mon[31461]: pgmap v738: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:18.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:18 smithi097 ceph-mon[28127]: pgmap v738: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:20.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:20 smithi111 ceph-mon[31001]: pgmap v739: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:20.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:20 smithi143 ceph-mon[31461]: pgmap v739: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:20.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:20 smithi097 ceph-mon[28127]: pgmap v739: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:22.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:22 smithi111 ceph-mon[31001]: pgmap v740: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:22.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:22 smithi143 ceph-mon[31461]: pgmap v740: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:22.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:22 smithi097 ceph-mon[28127]: pgmap v740: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:24.367 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:24 smithi097 ceph-mon[28127]: pgmap v741: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:24.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:24 smithi111 ceph-mon[31001]: pgmap v741: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:24.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:24 smithi143 ceph-mon[31461]: pgmap v741: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:26 smithi111 ceph-mon[31001]: pgmap v742: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:26.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:26 smithi143 ceph-mon[31461]: pgmap v742: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:26.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:26 smithi097 ceph-mon[28127]: pgmap v742: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:28 smithi111 ceph-mon[31001]: pgmap v743: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:28.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:28 smithi143 ceph-mon[31461]: pgmap v743: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:28.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:28 smithi097 ceph-mon[28127]: pgmap v743: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:30.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:30 smithi111 ceph-mon[31001]: pgmap v744: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:30.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:30 smithi143 ceph-mon[31461]: pgmap v744: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:30.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:30 smithi097 ceph-mon[28127]: pgmap v744: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:32 smithi111 ceph-mon[31001]: pgmap v745: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:32.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:32 smithi143 ceph-mon[31461]: pgmap v745: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:32.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:32 smithi097 ceph-mon[28127]: pgmap v745: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:34 smithi111 ceph-mon[31001]: pgmap v746: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:34.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:34 smithi143 ceph-mon[31461]: pgmap v746: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:34.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:34 smithi097 ceph-mon[28127]: pgmap v746: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:36.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:36 smithi111 ceph-mon[31001]: pgmap v747: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:36.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:36 smithi143 ceph-mon[31461]: pgmap v747: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:36 smithi097 ceph-mon[28127]: pgmap v747: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:38.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:38 smithi111 ceph-mon[31001]: pgmap v748: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:38.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:38 smithi143 ceph-mon[31461]: pgmap v748: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:38.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:38 smithi097 ceph-mon[28127]: pgmap v748: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:40.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:40 smithi143 ceph-mon[31461]: pgmap v749: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:40.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:40 smithi097 ceph-mon[28127]: pgmap v749: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:40.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:40 smithi111 ceph-mon[31001]: pgmap v749: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:42.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:42 smithi143 ceph-mon[31461]: pgmap v750: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:42.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:42 smithi097 ceph-mon[28127]: pgmap v750: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:42.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:42 smithi111 ceph-mon[31001]: pgmap v750: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:44.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:44 smithi143 ceph-mon[31461]: pgmap v751: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:44.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:44 smithi097 ceph-mon[28127]: pgmap v751: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:44.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:44 smithi111 ceph-mon[31001]: pgmap v751: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:45.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:45 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:45.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:45 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:45.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:45 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:47.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:46 smithi143 ceph-mon[31461]: pgmap v752: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:47.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:47.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:46 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:47.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:46 smithi097 ceph-mon[28127]: pgmap v752: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:47.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:47.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:46 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:46 smithi111 ceph-mon[31001]: pgmap v752: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:47.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:46 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: pgmap v753: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.143 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:48 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: pgmap v753: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:48 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: pgmap v753: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:49.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:48 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:50 smithi111 ceph-mon[31001]: pgmap v754: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:51.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:50 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:50 smithi143 ceph-mon[31461]: pgmap v754: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:51.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:50 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:50 smithi097 ceph-mon[28127]: pgmap v754: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:51.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:51.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:50 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:43:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:52 smithi111 ceph-mon[31001]: pgmap v755: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:53.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:52 smithi143 ceph-mon[31461]: pgmap v755: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:53.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:52 smithi097 ceph-mon[28127]: pgmap v755: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:54 smithi097 ceph-mon[28127]: pgmap v756: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:55.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:54 smithi111 ceph-mon[31001]: pgmap v756: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:55.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:54 smithi143 ceph-mon[31461]: pgmap v756: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:43:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:56 smithi097 ceph-mon[28127]: pgmap v757: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:56 smithi111 ceph-mon[31001]: pgmap v757: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:57.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:56 smithi143 ceph-mon[31461]: pgmap v757: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:43:59.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:43:58 smithi111 ceph-mon[31001]: pgmap v758: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:59.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:43:58 smithi143 ceph-mon[31461]: pgmap v758: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:43:59.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:43:58 smithi097 ceph-mon[28127]: pgmap v758: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:01.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:00 smithi111 ceph-mon[31001]: pgmap v759: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:01.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:00 smithi143 ceph-mon[31461]: pgmap v759: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:01.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:00 smithi097 ceph-mon[28127]: pgmap v759: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:02 smithi111 ceph-mon[31001]: pgmap v760: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:02 smithi143 ceph-mon[31461]: pgmap v760: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:03.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:02 smithi097 ceph-mon[28127]: pgmap v760: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:04 smithi111 ceph-mon[31001]: pgmap v761: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:04 smithi143 ceph-mon[31461]: pgmap v761: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:05.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:04 smithi097 ceph-mon[28127]: pgmap v761: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:07.297 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:06 smithi143 ceph-mon[31461]: pgmap v762: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:07.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:06 smithi097 ceph-mon[28127]: pgmap v762: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:07.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:06 smithi111 ceph-mon[31001]: pgmap v762: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:08 smithi143 ceph-mon[31461]: pgmap v763: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:08 smithi097 ceph-mon[28127]: pgmap v763: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:09.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:08 smithi111 ceph-mon[31001]: pgmap v763: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:11.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:10 smithi143 ceph-mon[31461]: pgmap v764: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:11.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:10 smithi097 ceph-mon[28127]: pgmap v764: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:11.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:10 smithi111 ceph-mon[31001]: pgmap v764: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:13.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:12 smithi143 ceph-mon[31461]: pgmap v765: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:12 smithi097 ceph-mon[28127]: pgmap v765: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:13.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:12 smithi111 ceph-mon[31001]: pgmap v765: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:14 smithi143 ceph-mon[31461]: pgmap v766: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:14 smithi097 ceph-mon[28127]: pgmap v766: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:15.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:14 smithi111 ceph-mon[31001]: pgmap v766: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:16 smithi143 ceph-mon[31461]: pgmap v767: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:16 smithi097 ceph-mon[28127]: pgmap v767: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:17.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:16 smithi111 ceph-mon[31001]: pgmap v767: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:19.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:19 smithi111 ceph-mon[31001]: pgmap v768: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:19 smithi143 ceph-mon[31461]: pgmap v768: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:19 smithi097 ceph-mon[28127]: pgmap v768: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:20.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:20 smithi097 ceph-mon[28127]: pgmap v769: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:20.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:20 smithi111 ceph-mon[31001]: pgmap v769: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:20.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:20 smithi143 ceph-mon[31461]: pgmap v769: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:22 smithi097 ceph-mon[28127]: pgmap v770: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:22.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:22 smithi111 ceph-mon[31001]: pgmap v770: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:22.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:22 smithi143 ceph-mon[31461]: pgmap v770: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:24.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:24 smithi111 ceph-mon[31001]: pgmap v771: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:24.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:24 smithi143 ceph-mon[31461]: pgmap v771: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:24 smithi097 ceph-mon[28127]: pgmap v771: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:26.365 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:26 smithi097 ceph-mon[28127]: pgmap v772: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:26.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:26 smithi111 ceph-mon[31001]: pgmap v772: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:26.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:26 smithi143 ceph-mon[31461]: pgmap v772: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:28.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:28 smithi111 ceph-mon[31001]: pgmap v773: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:28.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:28 smithi143 ceph-mon[31461]: pgmap v773: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:28.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:28 smithi097 ceph-mon[28127]: pgmap v773: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:30.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:30 smithi111 ceph-mon[31001]: pgmap v774: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:30.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:30 smithi143 ceph-mon[31461]: pgmap v774: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:30.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:30 smithi097 ceph-mon[28127]: pgmap v774: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:32 smithi111 ceph-mon[31001]: pgmap v775: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:32.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:32 smithi143 ceph-mon[31461]: pgmap v775: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:32.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:32 smithi097 ceph-mon[28127]: pgmap v775: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:34.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:34 smithi111 ceph-mon[31001]: pgmap v776: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:34.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:34 smithi143 ceph-mon[31461]: pgmap v776: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:34.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:34 smithi097 ceph-mon[28127]: pgmap v776: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:36.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:36 smithi111 ceph-mon[31001]: pgmap v777: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:36.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:36 smithi143 ceph-mon[31461]: pgmap v777: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:36 smithi097 ceph-mon[28127]: pgmap v777: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:38.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:38 smithi111 ceph-mon[31001]: pgmap v778: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:38.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:38 smithi143 ceph-mon[31461]: pgmap v778: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:38.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:38 smithi097 ceph-mon[28127]: pgmap v778: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:40.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:40 smithi111 ceph-mon[31001]: pgmap v779: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:40.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:40 smithi143 ceph-mon[31461]: pgmap v779: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:40.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:40 smithi097 ceph-mon[28127]: pgmap v779: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:42.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:42 smithi111 ceph-mon[31001]: pgmap v780: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:42.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:42 smithi143 ceph-mon[31461]: pgmap v780: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:42.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:42 smithi097 ceph-mon[28127]: pgmap v780: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:44.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:44 smithi111 ceph-mon[31001]: pgmap v781: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:44.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:44 smithi143 ceph-mon[31461]: pgmap v781: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:44.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:44 smithi097 ceph-mon[28127]: pgmap v781: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:44:46.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:46 smithi111 ceph-mon[31001]: pgmap v782: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:46.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:46 smithi143 ceph-mon[31461]: pgmap v782: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:46.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:46 smithi097 ceph-mon[28127]: pgmap v782: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:48.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:48 smithi143 ceph-mon[31461]: pgmap v783: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:48.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:48 smithi097 ceph-mon[28127]: pgmap v783: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:48.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:48 smithi111 ceph-mon[31001]: pgmap v783: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:44:50.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:50 smithi143 ceph-mon[31461]: pgmap v784: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:50 smithi097 ceph-mon[28127]: pgmap v784: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:50.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:50 smithi111 ceph-mon[31001]: pgmap v784: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:51.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:51 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:51.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:51 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:51.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:51 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:52.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:52 smithi143 ceph-mon[31461]: pgmap v785: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:52.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:52.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:52 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:44:52.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:52 smithi097 ceph-mon[28127]: pgmap v785: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:52.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:52.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:52 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:44:52.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:52 smithi111 ceph-mon[31001]: pgmap v785: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:44:52.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:52.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:52 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:44:54.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:54 smithi143 ceph-mon[31461]: pgmap v786: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:44:54.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:54 smithi097 ceph-mon[28127]: pgmap v786: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:44:54.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:54 smithi111 ceph-mon[31001]: pgmap v786: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:44:56.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:56 smithi143 ceph-mon[31461]: pgmap v787: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:44:56.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:56 smithi097 ceph-mon[28127]: pgmap v787: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:44:56.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:56 smithi111 ceph-mon[31001]: pgmap v787: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:44:58.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:44:58 smithi143 ceph-mon[31461]: pgmap v788: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:44:58.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:44:58 smithi097 ceph-mon[28127]: pgmap v788: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:44:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:44:58 smithi111 ceph-mon[31001]: pgmap v788: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:00.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:00 smithi143 ceph-mon[31461]: pgmap v789: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:45:00.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:00 smithi097 ceph-mon[28127]: pgmap v789: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:45:00.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:00 smithi111 ceph-mon[31001]: pgmap v789: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T21:45:02.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:02 smithi143 ceph-mon[31461]: pgmap v790: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:45:02.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:02 smithi097 ceph-mon[28127]: pgmap v790: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:45:02.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:02 smithi111 ceph-mon[31001]: pgmap v790: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T21:45:04.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:04 smithi143 ceph-mon[31461]: pgmap v791: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:45:04.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:04 smithi097 ceph-mon[28127]: pgmap v791: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:45:04.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:04 smithi111 ceph-mon[31001]: pgmap v791: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-26T21:45:06.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:06 smithi143 ceph-mon[31461]: pgmap v792: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:06.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:06 smithi097 ceph-mon[28127]: pgmap v792: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:06.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:06 smithi111 ceph-mon[31001]: pgmap v792: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:08.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:08 smithi143 ceph-mon[31461]: pgmap v793: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:08.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:08 smithi097 ceph-mon[28127]: pgmap v793: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:08.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:08 smithi111 ceph-mon[31001]: pgmap v793: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:10.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:10 smithi143 ceph-mon[31461]: pgmap v794: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:10.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:10 smithi097 ceph-mon[28127]: pgmap v794: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:10.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:10 smithi111 ceph-mon[31001]: pgmap v794: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:12.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:12 smithi143 ceph-mon[31461]: pgmap v795: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:12.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:12 smithi097 ceph-mon[28127]: pgmap v795: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:12.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:12 smithi111 ceph-mon[31001]: pgmap v795: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:14.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:14 smithi143 ceph-mon[31461]: pgmap v796: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:14.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:14 smithi097 ceph-mon[28127]: pgmap v796: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:14.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:14 smithi111 ceph-mon[31001]: pgmap v796: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:16.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:16 smithi143 ceph-mon[31461]: pgmap v797: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:16.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:16 smithi097 ceph-mon[28127]: pgmap v797: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:16.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:16 smithi111 ceph-mon[31001]: pgmap v797: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:18.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:18 smithi143 ceph-mon[31461]: pgmap v798: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:18.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:18 smithi097 ceph-mon[28127]: pgmap v798: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:18.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:18 smithi111 ceph-mon[31001]: pgmap v798: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:18.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:45:18.742+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4a8006110 con 0x7fa4d0104bc0 2024-04-26T21:45:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:45:18.743+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fa4a8006110 con 0x7fa4d0104bc0 2024-04-26T21:45:20.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:20 smithi143 ceph-mon[31461]: pgmap v799: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:20.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:20 smithi097 ceph-mon[28127]: pgmap v799: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:20.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:20 smithi111 ceph-mon[31001]: pgmap v799: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:22.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:22 smithi143 ceph-mon[31461]: pgmap v800: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:22.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:22 smithi097 ceph-mon[28127]: pgmap v800: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:22.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:22 smithi111 ceph-mon[31001]: pgmap v800: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:24.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:24 smithi143 ceph-mon[31461]: pgmap v801: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:24.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:24 smithi097 ceph-mon[28127]: pgmap v801: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:24.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:24 smithi111 ceph-mon[31001]: pgmap v801: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:26.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:26 smithi143 ceph-mon[31461]: pgmap v802: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:26.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:26 smithi097 ceph-mon[28127]: pgmap v802: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:26.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:26 smithi111 ceph-mon[31001]: pgmap v802: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:28.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:28 smithi143 ceph-mon[31461]: pgmap v803: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:28.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:28 smithi097 ceph-mon[28127]: pgmap v803: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:28.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:28 smithi111 ceph-mon[31001]: pgmap v803: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:45:30.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:30 smithi143 ceph-mon[31461]: pgmap v804: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:30.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:30 smithi097 ceph-mon[28127]: pgmap v804: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:30.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:30 smithi111 ceph-mon[31001]: pgmap v804: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:32.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:32 smithi143 ceph-mon[31461]: pgmap v805: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:32.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:32 smithi097 ceph-mon[28127]: pgmap v805: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:32.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:32 smithi111 ceph-mon[31001]: pgmap v805: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:34.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:34 smithi097 ceph-mon[28127]: pgmap v806: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:34.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:34 smithi111 ceph-mon[31001]: pgmap v806: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:34.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:34 smithi143 ceph-mon[31461]: pgmap v806: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:45:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:36 smithi097 ceph-mon[28127]: pgmap v807: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:36.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:36 smithi111 ceph-mon[31001]: pgmap v807: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:36.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:36 smithi143 ceph-mon[31461]: pgmap v807: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:38.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:38 smithi097 ceph-mon[28127]: pgmap v808: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:38.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:38 smithi111 ceph-mon[31001]: pgmap v808: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:38.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:38 smithi143 ceph-mon[31461]: pgmap v808: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:40.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:40 smithi111 ceph-mon[31001]: pgmap v809: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:40.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:40 smithi143 ceph-mon[31461]: pgmap v809: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:40.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:40 smithi097 ceph-mon[28127]: pgmap v809: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:42.597 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:42 smithi143 ceph-mon[31461]: pgmap v810: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:42.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:42 smithi111 ceph-mon[31001]: pgmap v810: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:42.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:42 smithi097 ceph-mon[28127]: pgmap v810: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:44.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:44 smithi111 ceph-mon[31001]: pgmap v811: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:44.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:44 smithi143 ceph-mon[31461]: pgmap v811: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:44.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:44 smithi097 ceph-mon[28127]: pgmap v811: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:46.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:46 smithi111 ceph-mon[31001]: pgmap v812: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:46.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:46 smithi143 ceph-mon[31461]: pgmap v812: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:46.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:46 smithi097 ceph-mon[28127]: pgmap v812: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-26T21:45:48.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:48 smithi111 ceph-mon[31001]: pgmap v813: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:48.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:48 smithi143 ceph-mon[31461]: pgmap v813: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:48 smithi097 ceph-mon[28127]: pgmap v813: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:50.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:50 smithi111 ceph-mon[31001]: pgmap v814: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:50.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:50 smithi143 ceph-mon[31461]: pgmap v814: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:50.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:50 smithi097 ceph-mon[28127]: pgmap v814: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:45:52.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:52 smithi111 ceph-mon[31001]: pgmap v815: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:52.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:52 smithi143 ceph-mon[31461]: pgmap v815: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:52.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:52 smithi097 ceph-mon[28127]: pgmap v815: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:53.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:53 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:53.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:53 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:53.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:53 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:54.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:54 smithi111 ceph-mon[31001]: pgmap v816: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:54.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:54.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:54.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:45:54.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:54 smithi143 ceph-mon[31461]: pgmap v816: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:54.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:54.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:54.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:45:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:54 smithi097 ceph-mon[28127]: pgmap v816: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:45:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:45:56.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:56 smithi111 ceph-mon[31001]: pgmap v817: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:56.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:56 smithi143 ceph-mon[31461]: pgmap v817: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:56.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:56 smithi097 ceph-mon[28127]: pgmap v817: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:45:58.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:45:58 smithi111 ceph-mon[31001]: pgmap v818: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:58.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:45:58 smithi143 ceph-mon[31461]: pgmap v818: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:45:58.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:45:58 smithi097 ceph-mon[28127]: pgmap v818: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:46:00.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:00 smithi111 ceph-mon[31001]: pgmap v819: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:00.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:00 smithi143 ceph-mon[31461]: pgmap v819: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:00.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:00 smithi097 ceph-mon[28127]: pgmap v819: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:02.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:02 smithi143 ceph-mon[31461]: pgmap v820: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:02.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:02 smithi097 ceph-mon[28127]: pgmap v820: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:02.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:02 smithi111 ceph-mon[31001]: pgmap v820: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:04.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:04 smithi143 ceph-mon[31461]: pgmap v821: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:04.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:04 smithi097 ceph-mon[28127]: pgmap v821: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:04.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:04 smithi111 ceph-mon[31001]: pgmap v821: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:06.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:06 smithi143 ceph-mon[31461]: pgmap v822: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:06.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:06 smithi097 ceph-mon[28127]: pgmap v822: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:06.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:06 smithi111 ceph-mon[31001]: pgmap v822: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:08.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:08 smithi143 ceph-mon[31461]: pgmap v823: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:08.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:08 smithi097 ceph-mon[28127]: pgmap v823: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:08.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:08 smithi111 ceph-mon[31001]: pgmap v823: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:10.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:10 smithi143 ceph-mon[31461]: pgmap v824: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:10.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:10 smithi097 ceph-mon[28127]: pgmap v824: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:10.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:10 smithi111 ceph-mon[31001]: pgmap v824: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:12.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:12 smithi143 ceph-mon[31461]: pgmap v825: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:12.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:12 smithi097 ceph-mon[28127]: pgmap v825: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:12.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:12 smithi111 ceph-mon[31001]: pgmap v825: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:14.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:14 smithi143 ceph-mon[31461]: pgmap v826: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:14.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:14 smithi097 ceph-mon[28127]: pgmap v826: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:14 smithi111 ceph-mon[31001]: pgmap v826: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:16.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:16 smithi143 ceph-mon[31461]: pgmap v827: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:16.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:16 smithi097 ceph-mon[28127]: pgmap v827: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:16.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:16 smithi111 ceph-mon[31001]: pgmap v827: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:18.697 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:18 smithi111 ceph-mon[31001]: pgmap v828: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:18.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:18 smithi143 ceph-mon[31461]: pgmap v828: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:18.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:18 smithi097 ceph-mon[28127]: pgmap v828: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:20.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:20 smithi143 ceph-mon[31461]: pgmap v829: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:20.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:20 smithi097 ceph-mon[28127]: pgmap v829: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:20.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:20 smithi111 ceph-mon[31001]: pgmap v829: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:22.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:22 smithi143 ceph-mon[31461]: pgmap v830: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:22.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:22 smithi097 ceph-mon[28127]: pgmap v830: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:22.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:22 smithi111 ceph-mon[31001]: pgmap v830: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:24.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:24 smithi143 ceph-mon[31461]: pgmap v831: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:24.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:24 smithi097 ceph-mon[28127]: pgmap v831: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:24.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:24 smithi111 ceph-mon[31001]: pgmap v831: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:26.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:26 smithi143 ceph-mon[31461]: pgmap v832: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:26.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:26 smithi097 ceph-mon[28127]: pgmap v832: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:26.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:26 smithi111 ceph-mon[31001]: pgmap v832: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:28.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:28 smithi143 ceph-mon[31461]: pgmap v833: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:28.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:28 smithi097 ceph-mon[28127]: pgmap v833: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:28.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:28 smithi111 ceph-mon[31001]: pgmap v833: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:30.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:30 smithi143 ceph-mon[31461]: pgmap v834: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:30.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:30 smithi097 ceph-mon[28127]: pgmap v834: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:30.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:30 smithi111 ceph-mon[31001]: pgmap v834: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:32.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:32 smithi143 ceph-mon[31461]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:32.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:32 smithi097 ceph-mon[28127]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:32.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:32 smithi111 ceph-mon[31001]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:34.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:34 smithi143 ceph-mon[31461]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:34.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:34 smithi097 ceph-mon[28127]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:34.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:34 smithi111 ceph-mon[31001]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:36.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:36 smithi143 ceph-mon[31461]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:36.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:36 smithi097 ceph-mon[28127]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:36.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:36 smithi111 ceph-mon[31001]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:38.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:38 smithi143 ceph-mon[31461]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:38.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:38 smithi097 ceph-mon[28127]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:38.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:38 smithi111 ceph-mon[31001]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:40.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:40 smithi143 ceph-mon[31461]: pgmap v839: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:40.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:40 smithi097 ceph-mon[28127]: pgmap v839: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:40.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:40 smithi111 ceph-mon[31001]: pgmap v839: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:42.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:42 smithi143 ceph-mon[31461]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:42.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:42 smithi097 ceph-mon[28127]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:42.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:42 smithi111 ceph-mon[31001]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:44.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:44 smithi143 ceph-mon[31461]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:46:44.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:44 smithi097 ceph-mon[28127]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:46:44.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:44 smithi111 ceph-mon[31001]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:46:46.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:46 smithi143 ceph-mon[31461]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:46.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:46 smithi097 ceph-mon[28127]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:46.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:46 smithi111 ceph-mon[31001]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:48.794 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:48 smithi143 ceph-mon[31461]: pgmap v843: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:46:48.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:48 smithi097 ceph-mon[28127]: pgmap v843: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:46:48.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:48 smithi111 ceph-mon[31001]: pgmap v843: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-26T21:46:50.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:50 smithi097 ceph-mon[28127]: pgmap v844: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:50.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:50 smithi111 ceph-mon[31001]: pgmap v844: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:51.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:50 smithi143 ceph-mon[31461]: pgmap v844: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-26T21:46:52.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:52 smithi097 ceph-mon[28127]: pgmap v845: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:52.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:52 smithi111 ceph-mon[31001]: pgmap v845: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:53.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:52 smithi143 ceph-mon[31461]: pgmap v845: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:54 smithi097 ceph-mon[28127]: pgmap v846: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:54.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:54 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:54.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:54 smithi111 ceph-mon[31001]: pgmap v846: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:54.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:54 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:55.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:54 smithi143 ceph-mon[31461]: pgmap v846: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:46:55.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:54 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:56.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:56 smithi111 ceph-mon[31001]: pgmap v847: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:56.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:56.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:56.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:46:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:56 smithi143 ceph-mon[31461]: pgmap v847: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:46:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:56 smithi097 ceph-mon[28127]: pgmap v847: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:46:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:46:58.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:46:58 smithi111 ceph-mon[31001]: pgmap v848: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:59.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:46:58 smithi143 ceph-mon[31461]: pgmap v848: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:46:59.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:46:58 smithi097 ceph-mon[28127]: pgmap v848: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:00.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:00 smithi111 ceph-mon[31001]: pgmap v849: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:01.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:00 smithi143 ceph-mon[31461]: pgmap v849: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:01.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:00 smithi097 ceph-mon[28127]: pgmap v849: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:02.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:02 smithi111 ceph-mon[31001]: pgmap v850: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:03.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:02 smithi143 ceph-mon[31461]: pgmap v850: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:02 smithi097 ceph-mon[28127]: pgmap v850: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:04.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:04 smithi111 ceph-mon[31001]: pgmap v851: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:05.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:04 smithi143 ceph-mon[31461]: pgmap v851: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:05.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:04 smithi097 ceph-mon[28127]: pgmap v851: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:06.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:06 smithi111 ceph-mon[31001]: pgmap v852: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:07.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:06 smithi143 ceph-mon[31461]: pgmap v852: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:07.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:06 smithi097 ceph-mon[28127]: pgmap v852: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:08.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:08 smithi111 ceph-mon[31001]: pgmap v853: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:09.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:08 smithi143 ceph-mon[31461]: pgmap v853: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:09.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:08 smithi097 ceph-mon[28127]: pgmap v853: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:10.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:10 smithi111 ceph-mon[31001]: pgmap v854: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:11.046 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:10 smithi143 ceph-mon[31461]: pgmap v854: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:11.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:10 smithi097 ceph-mon[28127]: pgmap v854: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:12.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:12 smithi111 ceph-mon[31001]: pgmap v855: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:13.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:12 smithi143 ceph-mon[31461]: pgmap v855: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:13.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:12 smithi097 ceph-mon[28127]: pgmap v855: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:14.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:14 smithi111 ceph-mon[31001]: pgmap v856: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:15.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:14 smithi143 ceph-mon[31461]: pgmap v856: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:15.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:14 smithi097 ceph-mon[28127]: pgmap v856: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:16.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:16 smithi111 ceph-mon[31001]: pgmap v857: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:17.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:16 smithi143 ceph-mon[31461]: pgmap v857: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:17.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:16 smithi097 ceph-mon[28127]: pgmap v857: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:18.892 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:18 smithi111 ceph-mon[31001]: pgmap v858: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:19.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:18 smithi143 ceph-mon[31461]: pgmap v858: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:19.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:18 smithi097 ceph-mon[28127]: pgmap v858: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:21.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:20 smithi143 ceph-mon[31461]: pgmap v859: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:21.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:20 smithi097 ceph-mon[28127]: pgmap v859: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:20 smithi111 ceph-mon[31001]: pgmap v859: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:23.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:22 smithi143 ceph-mon[31461]: pgmap v860: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:23.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:22 smithi097 ceph-mon[28127]: pgmap v860: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:23.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:22 smithi111 ceph-mon[31001]: pgmap v860: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:25.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:24 smithi143 ceph-mon[31461]: pgmap v861: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:25.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:24 smithi097 ceph-mon[28127]: pgmap v861: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:25.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:24 smithi111 ceph-mon[31001]: pgmap v861: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:27.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:26 smithi143 ceph-mon[31461]: pgmap v862: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:27.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:26 smithi097 ceph-mon[28127]: pgmap v862: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:27.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:26 smithi111 ceph-mon[31001]: pgmap v862: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:29.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:28 smithi143 ceph-mon[31461]: pgmap v863: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:29.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:28 smithi097 ceph-mon[28127]: pgmap v863: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:29.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:28 smithi111 ceph-mon[31001]: pgmap v863: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:31.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:30 smithi143 ceph-mon[31461]: pgmap v864: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:31.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:30 smithi097 ceph-mon[28127]: pgmap v864: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:31.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:30 smithi111 ceph-mon[31001]: pgmap v864: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:33.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:32 smithi143 ceph-mon[31461]: pgmap v865: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:33.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:32 smithi097 ceph-mon[28127]: pgmap v865: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:33.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:32 smithi111 ceph-mon[31001]: pgmap v865: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:35.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:34 smithi143 ceph-mon[31461]: pgmap v866: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:35.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:34 smithi097 ceph-mon[28127]: pgmap v866: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:35.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:34 smithi111 ceph-mon[31001]: pgmap v866: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:37.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:36 smithi143 ceph-mon[31461]: pgmap v867: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:36 smithi097 ceph-mon[28127]: pgmap v867: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:37.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:36 smithi111 ceph-mon[31001]: pgmap v867: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:39.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:38 smithi143 ceph-mon[31461]: pgmap v868: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:39.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:38 smithi097 ceph-mon[28127]: pgmap v868: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:39.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:38 smithi111 ceph-mon[31001]: pgmap v868: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:41.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:40 smithi143 ceph-mon[31461]: pgmap v869: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:41.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:40 smithi097 ceph-mon[28127]: pgmap v869: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:41.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:40 smithi111 ceph-mon[31001]: pgmap v869: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:43.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:42 smithi143 ceph-mon[31461]: pgmap v870: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:43.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:42 smithi097 ceph-mon[28127]: pgmap v870: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:43.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:42 smithi111 ceph-mon[31001]: pgmap v870: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:45.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:44 smithi143 ceph-mon[31461]: pgmap v871: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:45.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:44 smithi097 ceph-mon[28127]: pgmap v871: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:45.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:44 smithi111 ceph-mon[31001]: pgmap v871: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:47.045 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:46 smithi143 ceph-mon[31461]: pgmap v872: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:47.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:46 smithi097 ceph-mon[28127]: pgmap v872: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:47.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:46 smithi111 ceph-mon[31001]: pgmap v872: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:49.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:48 smithi143 ceph-mon[31461]: pgmap v873: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:49.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:48 smithi097 ceph-mon[28127]: pgmap v873: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:49.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:48 smithi111 ceph-mon[31001]: pgmap v873: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:51.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:50 smithi143 ceph-mon[31461]: pgmap v874: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:51.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:50 smithi097 ceph-mon[28127]: pgmap v874: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:51.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:50 smithi111 ceph-mon[31001]: pgmap v874: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:53.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:52 smithi143 ceph-mon[31461]: pgmap v875: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:53.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:52 smithi097 ceph-mon[28127]: pgmap v875: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:53.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:52 smithi111 ceph-mon[31001]: pgmap v875: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:55.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:54 smithi143 ceph-mon[31461]: pgmap v876: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:55.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:54 smithi097 ceph-mon[28127]: pgmap v876: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:55.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:54 smithi111 ceph-mon[31001]: pgmap v876: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:47:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:56 smithi143 ceph-mon[31461]: pgmap v877: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:57.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:56 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:56 smithi097 ceph-mon[28127]: pgmap v877: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:57.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:56 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:57.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:56 smithi111 ceph-mon[31001]: pgmap v877: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:47:57.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:56 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:58.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:57 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:47:58.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:58.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:58.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:57 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:47:58.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:58.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:57 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' 2024-04-26T21:47:59.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:47:58 smithi143 ceph-mon[31461]: pgmap v878: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:59.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:47:58 smithi097 ceph-mon[28127]: pgmap v878: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:47:59.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:47:58 smithi111 ceph-mon[31001]: pgmap v878: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:01.044 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:00 smithi143 ceph-mon[31461]: pgmap v879: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:01.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:00 smithi097 ceph-mon[28127]: pgmap v879: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:01.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:00 smithi111 ceph-mon[31001]: pgmap v879: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:03.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:02 smithi097 ceph-mon[28127]: pgmap v880: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:03.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:02 smithi111 ceph-mon[31001]: pgmap v880: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:03.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:02 smithi143 ceph-mon[31461]: pgmap v880: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:05.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:04 smithi097 ceph-mon[28127]: pgmap v881: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:05.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:04 smithi111 ceph-mon[31001]: pgmap v881: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:05.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:04 smithi143 ceph-mon[31461]: pgmap v881: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:07.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:06 smithi097 ceph-mon[28127]: pgmap v882: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:07.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:06 smithi111 ceph-mon[31001]: pgmap v882: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:07.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:06 smithi143 ceph-mon[31461]: pgmap v882: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:09.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:08 smithi111 ceph-mon[31001]: pgmap v883: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:09.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:08 smithi143 ceph-mon[31461]: pgmap v883: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:09.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:08 smithi097 ceph-mon[28127]: pgmap v883: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:11.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:10 smithi111 ceph-mon[31001]: pgmap v884: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:11.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:10 smithi143 ceph-mon[31461]: pgmap v884: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:11.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:10 smithi097 ceph-mon[28127]: pgmap v884: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:13.097 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:12 smithi143 ceph-mon[31461]: pgmap v885: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:13.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:12 smithi111 ceph-mon[31001]: pgmap v885: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:13.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:12 smithi097 ceph-mon[28127]: pgmap v885: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:15.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:14 smithi111 ceph-mon[31001]: pgmap v886: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:14 smithi143 ceph-mon[31461]: pgmap v886: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:15.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:14 smithi097 ceph-mon[28127]: pgmap v886: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:17.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:16 smithi111 ceph-mon[31001]: pgmap v887: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:17.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:16 smithi143 ceph-mon[31461]: pgmap v887: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:17.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:16 smithi097 ceph-mon[28127]: pgmap v887: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c6000640 1 -- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 msgr2=0x7fa4b40a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900021668) for more than 900000000 us, fault. 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 0x7fa4b40a61b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8031460 tx=0x7fa4b8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 msgr2=0x7fa4b40a61b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c7e00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40a3cf0 0x7fa4b40a61b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c7e00640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40adb80 0x7fa4b40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:18.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa4b40b0550 con 0x7fa4b40adb80 2024-04-26T21:48:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.654+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40adb80 0x7fa4b40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.656+0000 7fa4c6000640 1 --2- 172.21.15.97:0/2795231492 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa4b40adb80 0x7fa4b40aff70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8000c00 tx=0x7fa4b803f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:18.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.771+0000 7fa4c6a00640 1 -- 172.21.15.97:0/2795231492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4a80063d0 con 0x7fa4d0104bc0 2024-04-26T21:48:18.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:18.772+0000 7fa4c7e00640 1 -- 172.21.15.97:0/2795231492 <== mon.0 v2:172.21.15.97:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fa4a80063d0 con 0x7fa4d0104bc0 2024-04-26T21:48:18.873 INFO:teuthology.orchestra.run.smithi097.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-04-26T21:48:18.894 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-26T21:48:18.895 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-04-26T21:48:19.137 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6bcf2f8bdb994f7ab96583a11f36f637 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-04-26T21:48:19.139 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T21:48:19.149 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T21:48:19.149 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:48:19.149 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:19.153 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:18 smithi111 ceph-mon[31001]: pgmap v888: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:19.208 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:48:19.208 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:19.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:19.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:18 smithi143 ceph-mon[31461]: pgmap v888: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:19.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:18 smithi097 ceph-mon[28127]: pgmap v888: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-26T21:48:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 -- 172.21.15.97:0/1357755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 msgr2=0x7f4c40104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 --2- 172.21.15.97:0/1357755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c40104f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009900 tx=0x7f4c3002a410 comp rx=0 tx=0).stop 2024-04-26T21:48:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 -- 172.21.15.97:0/1357755937 shutdown_connections 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 --2- 172.21.15.97:0/1357755937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 0x7f4c4010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 --2- 172.21.15.97:0/1357755937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 0x7f4c401061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 --2- 172.21.15.97:0/1357755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c40104f90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 -- 172.21.15.97:0/1357755937 >> 172.21.15.97:0/1357755937 conn(0x7f4c40100120 msgr2=0x7f4c40102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 -- 172.21.15.97:0/1357755937 shutdown_connections 2024-04-26T21:48:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.470+0000 7f4c44e00640 1 -- 172.21.15.97:0/1357755937 wait complete. 2024-04-26T21:48:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 Processor -- start 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 -- start start 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 0x7f4c4019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 0x7f4c401a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40110540 con 0x7f4c40104b70 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401103c0 con 0x7f4c40106720 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.472+0000 7f4c44e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401106c0 con 0x7f4c40105d60 2024-04-26T21:48:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36678/0 (socket says 172.21.15.97:36678) 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 -- 172.21.15.97:0/3137640322 learned_addr learned my addr 172.21.15.97:0/3137640322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3d400640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 0x7f4c4019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3e800640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 0x7f4c401a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 -- 172.21.15.97:0/3137640322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 msgr2=0x7f4c4019b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 0x7f4c4019b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 -- 172.21.15.97:0/3137640322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 msgr2=0x7f4c401a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 0x7f4c401a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 -- 172.21.15.97:0/3137640322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c401a2930 con 0x7f4c40104b70 2024-04-26T21:48:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.473+0000 7f4c3de00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005ab0 tx=0x7f4c300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.474+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3003f030 con 0x7f4c40104b70 2024-04-26T21:48:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.474+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c30003c90 con 0x7f4c40104b70 2024-04-26T21:48:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.474+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401a2bc0 con 0x7f4c40104b70 2024-04-26T21:48:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.474+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30035070 con 0x7f4c40104b70 2024-04-26T21:48:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.474+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a3100 con 0x7f4c40104b70 2024-04-26T21:48:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.475+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4010bfa0 con 0x7f4c40104b70 2024-04-26T21:48:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.477+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4c30042030 con 0x7f4c40104b70 2024-04-26T21:48:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.478+0000 7f4c36a00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 0x7f4c240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.479+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f4c300f0fc0 con 0x7f4c40104b70 2024-04-26T21:48:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.479+0000 7f4c3d400640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 0x7f4c240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.479+0000 7f4c3d400640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 0x7f4c240a6180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005fd0 tx=0x7f4c2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.481+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4c300bd850 con 0x7f4c40104b70 2024-04-26T21:48:19.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.683+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c4002dd20 con 0x7f4c40104b70 2024-04-26T21:48:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.688+0000 7f4c36a00640 1 -- 172.21.15.97:0/3137640322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f4c300bd150 con 0x7f4c40104b70 2024-04-26T21:48:19.688 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:48:19.688 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 msgr2=0x7f4c240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 0x7f4c240a6180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005fd0 tx=0x7f4c2c005850 comp rx=0 tx=0).stop 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 msgr2=0x7f4c4019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4c30005ab0 tx=0x7f4c300042f0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 shutdown_connections 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c40106720 0x7f4c401a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c40105d60 0x7f4c4019b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f4c240a3cc0 0x7f4c240a6180 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 --2- 172.21.15.97:0/3137640322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40104b70 0x7f4c4019ad20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 >> 172.21.15.97:0/3137640322 conn(0x7f4c40100120 msgr2=0x7f4c40109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 shutdown_connections 2024-04-26T21:48:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.690+0000 7f4c44e00640 1 -- 172.21.15.97:0/3137640322 wait complete. 2024-04-26T21:48:19.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:19.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:19.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 -- 172.21.15.97:0/2506723046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18104b90 msgr2=0x7eff18104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 --2- 172.21.15.97:0/2506723046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18104b90 0x7eff18104fb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7eff08009900 tx=0x7eff0802a410 comp rx=0 tx=0).stop 2024-04-26T21:48:19.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 -- 172.21.15.97:0/2506723046 shutdown_connections 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 --2- 172.21.15.97:0/2506723046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18106740 0x7eff1810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 --2- 172.21.15.97:0/2506723046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18105d80 0x7eff18106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 --2- 172.21.15.97:0/2506723046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18104b90 0x7eff18104fb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 -- 172.21.15.97:0/2506723046 >> 172.21.15.97:0/2506723046 conn(0x7eff18100120 msgr2=0x7eff18102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.915+0000 7eff1d200640 1 -- 172.21.15.97:0/2506723046 shutdown_connections 2024-04-26T21:48:19.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.916+0000 7eff1d200640 1 -- 172.21.15.97:0/2506723046 wait complete. 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.916+0000 7eff1d200640 1 Processor -- start 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 -- start start 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18104b90 0x7eff1819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18106740 0x7eff181a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff18110560 con 0x7eff18105d80 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff181103e0 con 0x7eff18104b90 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff1d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7eff181106e0 con 0x7eff18106740 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36694/0 (socket says 172.21.15.97:36694) 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff0fe00640 1 -- 172.21.15.97:0/580876893 learned_addr learned my addr 172.21.15.97:0/580876893 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.917+0000 7eff15e00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18104b90 0x7eff1819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff16800640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18106740 0x7eff181a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 -- 172.21.15.97:0/580876893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18106740 msgr2=0x7eff181a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18106740 0x7eff181a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 -- 172.21.15.97:0/580876893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18104b90 msgr2=0x7eff1819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18104b90 0x7eff1819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 -- 172.21.15.97:0/580876893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff181a2900 con 0x7eff18105d80 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0fe00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7eff0000d770 tx=0x7eff0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00017070 con 0x7eff18105d80 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff000054b0 con 0x7eff18105d80 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff181a2bf0 con 0x7eff18105d80 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff181a30f0 con 0x7eff18105d80 2024-04-26T21:48:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.918+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00004c60 con 0x7eff18105d80 2024-04-26T21:48:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.920+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1810c020 con 0x7eff18105d80 2024-04-26T21:48:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.922+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff00004e00 con 0x7eff18105d80 2024-04-26T21:48:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.923+0000 7eff0ea00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 0x7efefc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.923+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7eff000d0100 con 0x7eff18105d80 2024-04-26T21:48:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.924+0000 7eff15e00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 0x7efefc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.924+0000 7eff15e00640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 0x7efefc0a6100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff080057c0 tx=0x7eff080095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:19.928+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff0009c990 con 0x7eff18105d80 2024-04-26T21:48:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:19 smithi097 ceph-mon[28127]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:20.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:19 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3137640322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:20.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.134+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff1802dcd0 con 0x7eff18105d80 2024-04-26T21:48:20.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.134+0000 7eff0ea00640 1 -- 172.21.15.97:0/580876893 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7eff0009c290 con 0x7eff18105d80 2024-04-26T21:48:20.134 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 msgr2=0x7efefc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 0x7efefc0a6100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff080057c0 tx=0x7eff080095f0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 msgr2=0x7eff1819b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7eff0000d770 tx=0x7eff0000dc30 comp rx=0 tx=0).stop 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 shutdown_connections 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff18106740 0x7eff181a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7efefc0a3c40 0x7efefc0a6100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18105d80 0x7eff1819b230 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 --2- 172.21.15.97:0/580876893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff18104b90 0x7eff1819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 >> 172.21.15.97:0/580876893 conn(0x7eff18100120 msgr2=0x7eff181091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:20.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 shutdown_connections 2024-04-26T21:48:20.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.137+0000 7eff1d200640 1 -- 172.21.15.97:0/580876893 wait complete. 2024-04-26T21:48:20.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:19 smithi111 ceph-mon[31001]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:20.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:19 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3137640322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:20.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:19 smithi143 ceph-mon[31461]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:20.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:19 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3137640322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 -- 172.21.15.97:0/1114235804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc10a980 msgr2=0x7f71cc1112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 --2- 172.21.15.97:0/1114235804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc10a980 0x7f71cc1112a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009ca0 tx=0x7f71c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 -- 172.21.15.97:0/1114235804 shutdown_connections 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 --2- 172.21.15.97:0/1114235804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc10a980 0x7f71cc1112a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 --2- 172.21.15.97:0/1114235804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc109fc0 0x7f71cc10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 --2- 172.21.15.97:0/1114235804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 0x7f71cc109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 -- 172.21.15.97:0/1114235804 >> 172.21.15.97:0/1114235804 conn(0x7f71cc077e00 msgr2=0x7f71cc07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:20.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 -- 172.21.15.97:0/1114235804 shutdown_connections 2024-04-26T21:48:20.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.357+0000 7f71cb200640 1 -- 172.21.15.97:0/1114235804 wait complete. 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.358+0000 7f71cb200640 1 Processor -- start 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.358+0000 7f71cb200640 1 -- start start 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.358+0000 7f71cb200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 0x7f71cc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.358+0000 7f71cb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71cb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc10a980 0x7f71cc1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71cb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1148d0 con 0x7f71cc109fc0 2024-04-26T21:48:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71cb200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc114750 con 0x7f71cc108e70 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71cb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc114a50 con 0x7f71cc10a980 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36714/0 (socket says 172.21.15.97:36714) 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 -- 172.21.15.97:0/4042023773 learned_addr learned my addr 172.21.15.97:0/4042023773 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9e00640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 0x7f71cc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71ca800640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc10a980 0x7f71cc1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 -- 172.21.15.97:0/4042023773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc10a980 msgr2=0x7f71cc1a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc10a980 0x7f71cc1a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 -- 172.21.15.97:0/4042023773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 msgr2=0x7f71cc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 0x7f71cc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.359+0000 7f71c9400640 1 -- 172.21.15.97:0/4042023773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1a6b40 con 0x7f71cc109fc0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71c9400640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d950 tx=0x7f71b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017070 con 0x7f71cc109fc0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b8009c30 con 0x7f71cc109fc0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1a6e30 con 0x7f71cc109fc0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800f620 con 0x7f71cc109fc0 2024-04-26T21:48:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.360+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a72a0 con 0x7f71cc109fc0 2024-04-26T21:48:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.362+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc110120 con 0x7f71cc109fc0 2024-04-26T21:48:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.364+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71b801a070 con 0x7f71cc109fc0 2024-04-26T21:48:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.366+0000 7f71c2a00640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 0x7f71b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.367+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f71b80d00b0 con 0x7f71cc109fc0 2024-04-26T21:48:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.367+0000 7f71c9e00640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 0x7f71b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.367+0000 7f71c9e00640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 0x7f71b00a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005fd0 tx=0x7f71bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:20.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.371+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f71b8009de0 con 0x7f71cc109fc0 2024-04-26T21:48:20.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.576+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71cc02dca0 con 0x7f71cc109fc0 2024-04-26T21:48:20.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.577+0000 7f71c2a00640 1 -- 172.21.15.97:0/4042023773 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f71b809c980 con 0x7f71cc109fc0 2024-04-26T21:48:20.576 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:48:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.579+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 msgr2=0x7f71b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.579+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 0x7f71b00a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005fd0 tx=0x7f71bc005850 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.579+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 msgr2=0x7f71cc19f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.579+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d950 tx=0x7f71b800de10 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 shutdown_connections 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc10a980 0x7f71cc1a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f71b00a3c40 0x7f71b00a6100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc109fc0 0x7f71cc19f4b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 --2- 172.21.15.97:0/4042023773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc108e70 0x7f71cc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 >> 172.21.15.97:0/4042023773 conn(0x7f71cc077e00 msgr2=0x7f71cc10d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 shutdown_connections 2024-04-26T21:48:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.580+0000 7f71cb200640 1 -- 172.21.15.97:0/4042023773 wait complete. 2024-04-26T21:48:20.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 -- 172.21.15.97:0/3082405636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 msgr2=0x7fa64810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 --2- 172.21.15.97:0/3082405636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64810a600 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa634009900 tx=0x7fa63402a410 comp rx=0 tx=0).stop 2024-04-26T21:48:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 -- 172.21.15.97:0/3082405636 shutdown_connections 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 --2- 172.21.15.97:0/3082405636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 0x7fa6481113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 --2- 172.21.15.97:0/3082405636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64810a600 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 --2- 172.21.15.97:0/3082405636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa6481093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 -- 172.21.15.97:0/3082405636 >> 172.21.15.97:0/3082405636 conn(0x7fa648077f30 msgr2=0x7fa64807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.802+0000 7fa650000640 1 -- 172.21.15.97:0/3082405636 shutdown_connections 2024-04-26T21:48:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.803+0000 7fa650000640 1 -- 172.21.15.97:0/3082405636 wait complete. 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 Processor -- start 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 -- start start 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa64819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 0x7fa6481a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481149d0 con 0x7fa64810a180 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa648114850 con 0x7fa64810ab40 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa650000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa648114b50 con 0x7fa648108f90 2024-04-26T21:48:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.804+0000 7fa64cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32974/0 (socket says 172.21.15.97:32974) 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 -- 172.21.15.97:0/3561177383 learned_addr learned my addr 172.21.15.97:0/3561177383 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64d600640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa64819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64e000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 0x7fa6481a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 -- 172.21.15.97:0/3561177383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 msgr2=0x7fa64819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa64819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 -- 172.21.15.97:0/3561177383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 msgr2=0x7fa6481a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:20.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 0x7fa6481a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 -- 172.21.15.97:0/3561177383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6481a6cd0 con 0x7fa64810a180 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64d600640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa64819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.805+0000 7fa64cc00640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa634031460 tx=0x7fa634003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.806+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63403f070 con 0x7fa64810a180 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.806+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa634031da0 con 0x7fa64810a180 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.806+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6481a6f60 con 0x7fa64810a180 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.806+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634035030 con 0x7fa64810a180 2024-04-26T21:48:20.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.806+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481a73c0 con 0x7fa64810a180 2024-04-26T21:48:20.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.807+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa648110320 con 0x7fa64810a180 2024-04-26T21:48:20.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.809+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa634042070 con 0x7fa64810a180 2024-04-26T21:48:20.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.810+0000 7fa646000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 0x7fa6300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:20.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.811+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fa6340f15a0 con 0x7fa64810a180 2024-04-26T21:48:20.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.811+0000 7fa64d600640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 0x7fa6300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:20.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.812+0000 7fa64d600640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 0x7fa6300a6100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0058c0 tx=0x7fa63c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:20.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:20.813+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6340bde30 con 0x7fa64810a180 2024-04-26T21:48:21.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.020+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa64802dd20 con 0x7fa64810a180 2024-04-26T21:48:21.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.022+0000 7fa646000640 1 -- 172.21.15.97:0/3561177383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fa6340bd730 con 0x7fa64810a180 2024-04-26T21:48:21.021 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:48:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 msgr2=0x7fa6300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 0x7fa6300a6100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0058c0 tx=0x7fa63c005850 comp rx=0 tx=0).stop 2024-04-26T21:48:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 msgr2=0x7fa64819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa634031460 tx=0x7fa634003c80 comp rx=0 tx=0).stop 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 shutdown_connections 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64810ab40 0x7fa6481a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fa6300a3c40 0x7fa6300a6100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa64810a180 0x7fa64819f5f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 --2- 172.21.15.97:0/3561177383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa648108f90 0x7fa64819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.024+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 >> 172.21.15.97:0/3561177383 conn(0x7fa648077f30 msgr2=0x7fa64810d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.025+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 shutdown_connections 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.025+0000 7fa650000640 1 -- 172.21.15.97:0/3561177383 wait complete. 2024-04-26T21:48:21.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:20 smithi111 ceph-mon[31001]: pgmap v889: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:20 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/580876893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:20 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4042023773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 -- 172.21.15.97:0/3552651507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 msgr2=0x7f2710103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 --2- 172.21.15.97:0/3552651507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f2710103dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009900 tx=0x7f26fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 -- 172.21.15.97:0/3552651507 shutdown_connections 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 --2- 172.21.15.97:0/3552651507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f271010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 --2- 172.21.15.97:0/3552651507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f2710103dd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 --2- 172.21.15.97:0/3552651507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 0x7f2710102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 -- 172.21.15.97:0/3552651507 >> 172.21.15.97:0/3552651507 conn(0x7f27100fdcb0 msgr2=0x7f2710100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 -- 172.21.15.97:0/3552651507 shutdown_connections 2024-04-26T21:48:21.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.234+0000 7f2716c00640 1 -- 172.21.15.97:0/3552651507 wait complete. 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 Processor -- start 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 -- start start 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 0x7f271019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f27101a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f271010e170 con 0x7f2710103950 2024-04-26T21:48:21.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f271010dff0 con 0x7f2710102760 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.235+0000 7f2716c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f271010e2f0 con 0x7f2710104310 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32992/0 (socket says 172.21.15.97:32992) 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 -- 172.21.15.97:0/3055575843 learned_addr learned my addr 172.21.15.97:0/3055575843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2715800640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 0x7f271019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2716200640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f27101a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 -- 172.21.15.97:0/3055575843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 msgr2=0x7f27101a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f27101a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 -- 172.21.15.97:0/3055575843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 msgr2=0x7f271019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 0x7f271019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2714e00640 1 -- 172.21.15.97:0/3055575843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101a25f0 con 0x7f2710103950 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.236+0000 7f2716200640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f27101a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f2714e00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26fc031460 tx=0x7f26fc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc03f070 con 0x7f2710103950 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26fc031da0 con 0x7f2710103950 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27101a2880 con 0x7f2710103950 2024-04-26T21:48:21.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc035030 con 0x7f2710103950 2024-04-26T21:48:21.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.237+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a2d70 con 0x7f2710103950 2024-04-26T21:48:21.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.238+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27101a2fd0 con 0x7f2710103950 2024-04-26T21:48:21.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.240+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f26fc042070 con 0x7f2710103950 2024-04-26T21:48:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.241+0000 7f270e000640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 0x7f26f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.241+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f26fc0f0e90 con 0x7f2710103950 2024-04-26T21:48:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.242+0000 7f2715800640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 0x7f26f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.242+0000 7f2715800640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 0x7f26f80a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27101037b0 tx=0x7f2704007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:21.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.246+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26fc0bd820 con 0x7f2710103950 2024-04-26T21:48:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:20 smithi143 ceph-mon[31461]: pgmap v889: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:20 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/580876893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:20 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4042023773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:20 smithi097 ceph-mon[28127]: pgmap v889: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:21.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:20 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/580876893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:20 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4042023773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.472+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27101a31a0 con 0x7f2710103950 2024-04-26T21:48:21.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.473+0000 7f270e000640 1 -- 172.21.15.97:0/3055575843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f26fc0bd120 con 0x7f2710103950 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/117675142 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1403922799 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3008702290 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2383506754 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/2998338527 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1403922799 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:21.473 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3810497084 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2341580572 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3660896935 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/2998338527 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:21.474 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.475+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 msgr2=0x7f26f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.475+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 0x7f26f80a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27101037b0 tx=0x7f2704007600 comp rx=0 tx=0).stop 2024-04-26T21:48:21.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.475+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 msgr2=0x7f271019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.475+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26fc031460 tx=0x7f26fc003d50 comp rx=0 tx=0).stop 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 shutdown_connections 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2710104310 0x7f27101a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f26f80a3c40 0x7f26f80a6100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2710103950 0x7f271019af10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 --2- 172.21.15.97:0/3055575843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2710102760 0x7f271019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 >> 172.21.15.97:0/3055575843 conn(0x7f27100fdcb0 msgr2=0x7f2710074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 shutdown_connections 2024-04-26T21:48:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.476+0000 7f2716c00640 1 -- 172.21.15.97:0/3055575843 wait complete. 2024-04-26T21:48:21.493 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T21:48:21.494 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:48:21.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 -- 172.21.15.97:0/1549864747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c104b90 msgr2=0x7f673c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 --2- 172.21.15.97:0/1549864747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c104b90 0x7f673c104fb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f672c009900 tx=0x7f672c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 -- 172.21.15.97:0/1549864747 shutdown_connections 2024-04-26T21:48:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 --2- 172.21.15.97:0/1549864747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 --2- 172.21.15.97:0/1549864747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c105d80 0x7f673c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 --2- 172.21.15.97:0/1549864747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c104b90 0x7f673c104fb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 -- 172.21.15.97:0/1549864747 >> 172.21.15.97:0/1549864747 conn(0x7f673c100120 msgr2=0x7f673c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 -- 172.21.15.97:0/1549864747 shutdown_connections 2024-04-26T21:48:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.790+0000 7f6742600640 1 -- 172.21.15.97:0/1549864747 wait complete. 2024-04-26T21:48:21.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.791+0000 7f6742600640 1 Processor -- start 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.791+0000 7f6742600640 1 -- start start 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 0x7f673c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c1a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1105b0 con 0x7f673c105d80 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f673c110430 con 0x7f673c106740 2024-04-26T21:48:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6742600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f673c110730 con 0x7f673c104b90 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6733e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6733e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33022/0 (socket says 172.21.15.97:33022) 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f6733e00640 1 -- 172.21.15.97:0/4208220631 learned_addr learned my addr 172.21.15.97:0/4208220631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f673b400640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 0x7f673c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.792+0000 7f673be00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c1a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 -- 172.21.15.97:0/4208220631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 msgr2=0x7f673c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 0x7f673c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 -- 172.21.15.97:0/4208220631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 msgr2=0x7f673c1a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:21.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c1a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 -- 172.21.15.97:0/4208220631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a2880 con 0x7f673c105d80 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f673b400640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 0x7f673c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f673be00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c1a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6733e00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f672400d770 tx=0x7f672400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017070 con 0x7f673c105d80 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67240054b0 con 0x7f673c105d80 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a2b70 con 0x7f673c105d80 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724004c60 con 0x7f673c105d80 2024-04-26T21:48:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.793+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a3070 con 0x7f673c105d80 2024-04-26T21:48:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.794+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c10bf30 con 0x7f673c105d80 2024-04-26T21:48:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.796+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6724004e00 con 0x7f673c105d80 2024-04-26T21:48:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.798+0000 7f6738c00640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.798+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f67240d0550 con 0x7f673c105d80 2024-04-26T21:48:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.798+0000 7f673b400640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.799+0000 7f673b400640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 0x7f67200a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f672c006010 tx=0x7f672c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.801+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f672409cde0 con 0x7f673c105d80 2024-04-26T21:48:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.994+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f673c02dca0 con 0x7f673c105d80 2024-04-26T21:48:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:21.998+0000 7f6738c00640 1 -- 172.21.15.97:0/4208220631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f673c02dca0 con 0x7f673c105d80 2024-04-26T21:48:21.999 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:48:21.999 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 msgr2=0x7f67200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 0x7f67200a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f672c006010 tx=0x7f672c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 msgr2=0x7f673c19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f672400d770 tx=0x7f672400dc30 comp rx=0 tx=0).stop 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 shutdown_connections 2024-04-26T21:48:22.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f673c106740 0x7f673c1a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c105d80 0x7f673c19b1b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 --2- 172.21.15.97:0/4208220631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f673c104b90 0x7f673c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 >> 172.21.15.97:0/4208220631 conn(0x7f673c100120 msgr2=0x7f673c109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 shutdown_connections 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.001+0000 7f6742600640 1 -- 172.21.15.97:0/4208220631 wait complete. 2024-04-26T21:48:22.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:24:36.675748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:30:51.224775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:30:52.223998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:48:22.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:48:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:21 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3561177383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:22.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:21 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3055575843' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:22.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 -- 172.21.15.97:0/2205481240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c10a780 msgr2=0x7fe26c10cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 --2- 172.21.15.97:0/2205481240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c10a780 0x7fe26c10cc10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009ca0 tx=0x7fe25c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 -- 172.21.15.97:0/2205481240 shutdown_connections 2024-04-26T21:48:22.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 --2- 172.21.15.97:0/2205481240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c10a780 0x7fe26c10cc10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 --2- 172.21.15.97:0/2205481240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c075390 0x7fe26c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 --2- 172.21.15.97:0/2205481240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 0x7fe26c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 -- 172.21.15.97:0/2205481240 >> 172.21.15.97:0/2205481240 conn(0x7fe26c0fde30 msgr2=0x7fe26c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 -- 172.21.15.97:0/2205481240 shutdown_connections 2024-04-26T21:48:22.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.223+0000 7fe270a00640 1 -- 172.21.15.97:0/2205481240 wait complete. 2024-04-26T21:48:22.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.224+0000 7fe270a00640 1 Processor -- start 2024-04-26T21:48:22.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.224+0000 7fe270a00640 1 -- start start 2024-04-26T21:48:22.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 0x7fe26c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c10a780 0x7fe26c1a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c110460 con 0x7fe26c075390 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1102e0 con 0x7fe26c076970 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe270a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c1105e0 con 0x7fe26c10a780 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe26b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe26b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33034/0 (socket says 172.21.15.97:33034) 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe26b400640 1 -- 172.21.15.97:0/1408887032 learned_addr learned my addr 172.21.15.97:0/1408887032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:22.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe26aa00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 0x7fe26c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.225+0000 7fe26be00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c10a780 0x7fe26c1a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 -- 172.21.15.97:0/1408887032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c10a780 msgr2=0x7fe26c1a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c10a780 0x7fe26c1a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 -- 172.21.15.97:0/1408887032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 msgr2=0x7fe26c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 0x7fe26c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 -- 172.21.15.97:0/1408887032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1a2780 con 0x7fe26c075390 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe26b400640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d950 tx=0x7fe26400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264017070 con 0x7fe26c075390 2024-04-26T21:48:22.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe264009c30 con 0x7fe26c075390 2024-04-26T21:48:22.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c1a2a70 con 0x7fe26c075390 2024-04-26T21:48:22.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26400f620 con 0x7fe26c075390 2024-04-26T21:48:22.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.226+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a2fc0 con 0x7fe26c075390 2024-04-26T21:48:22.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.228+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c1094b0 con 0x7fe26c075390 2024-04-26T21:48:22.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.229+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe264010460 con 0x7fe26c075390 2024-04-26T21:48:22.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.231+0000 7fe263e00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.231+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7fe2640d0190 con 0x7fe26c075390 2024-04-26T21:48:22.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.231+0000 7fe26aa00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.232+0000 7fe26aa00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe26c19c070 tx=0x7fe25400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:22.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.235+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe26409ca20 con 0x7fe26c075390 2024-04-26T21:48:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:21 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3561177383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:22.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:21 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3055575843' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:21 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3561177383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:22.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:21 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3055575843' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:22.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.437+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe26c063530 con 0x7fe26c075390 2024-04-26T21:48:22.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.438+0000 7fe263e00640 1 -- 172.21.15.97:0/1408887032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fe264009de0 con 0x7fe26c075390 2024-04-26T21:48:22.438 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 msgr2=0x7fe2500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe26c19c070 tx=0x7fe25400b390 comp rx=0 tx=0).stop 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 msgr2=0x7fe26c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d950 tx=0x7fe26400de10 comp rx=0 tx=0).stop 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 shutdown_connections 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe26c10a780 0x7fe26c1a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe26c076970 0x7fe26c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 --2- 172.21.15.97:0/1408887032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe26c075390 0x7fe26c19ab60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 >> 172.21.15.97:0/1408887032 conn(0x7fe26c0fde30 msgr2=0x7fe26c106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:22.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 shutdown_connections 2024-04-26T21:48:22.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.441+0000 7fe270a00640 1 -- 172.21.15.97:0/1408887032 wait complete. 2024-04-26T21:48:22.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:48:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 -- 172.21.15.97:0/1086863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad0102840 msgr2=0x7f8ad0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 --2- 172.21.15.97:0/1086863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad0102840 0x7f8ad0102c60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009900 tx=0x7f8ac402a410 comp rx=0 tx=0).stop 2024-04-26T21:48:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 -- 172.21.15.97:0/1086863173 shutdown_connections 2024-04-26T21:48:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 --2- 172.21.15.97:0/1086863173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad01043f0 0x7f8ad010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 --2- 172.21.15.97:0/1086863173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 0x7f8ad0103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 --2- 172.21.15.97:0/1086863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad0102840 0x7f8ad0102c60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 -- 172.21.15.97:0/1086863173 >> 172.21.15.97:0/1086863173 conn(0x7f8ad00fddd0 msgr2=0x7f8ad01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 -- 172.21.15.97:0/1086863173 shutdown_connections 2024-04-26T21:48:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.659+0000 7f8ad7000640 1 -- 172.21.15.97:0/1086863173 wait complete. 2024-04-26T21:48:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.660+0000 7f8ad7000640 1 Processor -- start 2024-04-26T21:48:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 -- start start 2024-04-26T21:48:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad0102840 0x7f8ad019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 0x7f8ad019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e270 con 0x7f8ad01043f0 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e0f0 con 0x7f8ad0103a30 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e3f0 con 0x7f8ad0102840 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33056/0 (socket says 172.21.15.97:33056) 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad6600640 1 -- 172.21.15.97:0/3709442748 learned_addr learned my addr 172.21.15.97:0/3709442748 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad5200640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 0x7f8ad019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.661+0000 7f8ad5c00640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad0102840 0x7f8ad019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 -- 172.21.15.97:0/3709442748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad0102840 msgr2=0x7f8ad019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad0102840 0x7f8ad019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 -- 172.21.15.97:0/3709442748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 msgr2=0x7f8ad019b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 0x7f8ad019b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 -- 172.21.15.97:0/3709442748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad01a27e0 con 0x7f8ad01043f0 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad6600640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800eaa0 tx=0x7f8ac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:22.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac800cdb0 con 0x7f8ad01043f0 2024-04-26T21:48:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ac8005020 con 0x7f8ad01043f0 2024-04-26T21:48:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80116d0 con 0x7f8ad01043f0 2024-04-26T21:48:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad01a2ad0 con 0x7f8ad01043f0 2024-04-26T21:48:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.662+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad01a3020 con 0x7f8ad01043f0 2024-04-26T21:48:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.663+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad01a3250 con 0x7f8ad01043f0 2024-04-26T21:48:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.665+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ac80051d0 con 0x7f8ad01043f0 2024-04-26T21:48:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.667+0000 7f8ace000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 0x7f8ab80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.667+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f8ac80d0a90 con 0x7f8ad01043f0 2024-04-26T21:48:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.667+0000 7f8ad5c00640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 0x7f8ab80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.668+0000 7f8ad5c00640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 0x7f8ab80a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40057c0 tx=0x7f8ac4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.669+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8ac8016030 con 0x7f8ad01043f0 2024-04-26T21:48:22.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.870+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ad01a3420 con 0x7f8ad01043f0 2024-04-26T21:48:22.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.871+0000 7f8ace000640 1 -- 172.21.15.97:0/3709442748 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f8ac809d420 con 0x7f8ad01043f0 2024-04-26T21:48:22.870 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-26T21:48:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.873+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 msgr2=0x7f8ab80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.873+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 0x7f8ab80a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40057c0 tx=0x7f8ac4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.873+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 msgr2=0x7f8ad01a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.873+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800eaa0 tx=0x7f8ac800ef60 comp rx=0 tx=0).stop 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 shutdown_connections 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f8ab80a3c40 0x7f8ab80a6100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad01043f0 0x7f8ad01a2170 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad0103a30 0x7f8ad019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 --2- 172.21.15.97:0/3709442748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ad0102840 0x7f8ad019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 >> 172.21.15.97:0/3709442748 conn(0x7f8ad00fddd0 msgr2=0x7f8ad0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 shutdown_connections 2024-04-26T21:48:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:22.874+0000 7f8ad7000640 1 -- 172.21.15.97:0/3709442748 wait complete. 2024-04-26T21:48:22.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:48:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 -- 172.21.15.97:0/75890453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c078530 msgr2=0x7f693c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 --2- 172.21.15.97:0/75890453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c078530 0x7f693c078990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6928009900 tx=0x7f692802a410 comp rx=0 tx=0).stop 2024-04-26T21:48:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 -- 172.21.15.97:0/75890453 shutdown_connections 2024-04-26T21:48:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 --2- 172.21.15.97:0/75890453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 --2- 172.21.15.97:0/75890453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c078530 0x7f693c078990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 --2- 172.21.15.97:0/75890453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c0772f0 0x7f693c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 -- 172.21.15.97:0/75890453 >> 172.21.15.97:0/75890453 conn(0x7f693c100120 msgr2=0x7f693c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 -- 172.21.15.97:0/75890453 shutdown_connections 2024-04-26T21:48:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.102+0000 7f6943600640 1 -- 172.21.15.97:0/75890453 wait complete. 2024-04-26T21:48:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.103+0000 7f6943600640 1 Processor -- start 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.103+0000 7f6943600640 1 -- start start 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 0x7f693c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c1a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1149c0 con 0x7f693c0772f0 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f693c114840 con 0x7f693c078ed0 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6943600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f693c114b40 con 0x7f693c078530 2024-04-26T21:48:23.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33078/0 (socket says 172.21.15.97:33078) 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 -- 172.21.15.97:0/3081610495 learned_addr learned my addr 172.21.15.97:0/3081610495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6941600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c1a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 -- 172.21.15.97:0/3081610495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 msgr2=0x7f693c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f693be00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 0x7f693c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 0x7f693c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 -- 172.21.15.97:0/3081610495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 msgr2=0x7f693c1a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c1a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6940c00640 1 -- 172.21.15.97:0/3081610495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1a6d20 con 0x7f693c0772f0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f6941600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c1a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.104+0000 7f693be00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 0x7f693c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6940c00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f693000d770 tx=0x7f693000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017070 con 0x7f693c0772f0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69300054b0 con 0x7f693c0772f0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a6fb0 con 0x7f693c0772f0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930004c60 con 0x7f693c0772f0 2024-04-26T21:48:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.105+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a7420 con 0x7f693c0772f0 2024-04-26T21:48:23.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.106+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c111790 con 0x7f693c0772f0 2024-04-26T21:48:23.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.108+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6930004e00 con 0x7f693c0772f0 2024-04-26T21:48:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.109+0000 7f6939600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 0x7f69240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.110+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f69300cff90 con 0x7f693c0772f0 2024-04-26T21:48:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.110+0000 7f693be00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 0x7f69240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.111+0000 7f693be00640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 0x7f69240a6100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f692802a930 tx=0x7f6928000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.114+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f693009c920 con 0x7f693c0772f0 2024-04-26T21:48:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:22 smithi111 ceph-mon[31001]: pgmap v890: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:22 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/4208220631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:22 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1408887032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:22 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3709442748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:22 smithi143 ceph-mon[31461]: pgmap v890: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:23.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:22 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/4208220631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:23.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:22 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1408887032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:22 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3709442748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:22 smithi097 ceph-mon[28127]: pgmap v890: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T21:48:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/4208220631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1408887032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:22 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3709442748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:48:23.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.314+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f693c0635d0 con 0x7f693c0772f0 2024-04-26T21:48:23.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.315+0000 7f6939600640 1 -- 172.21.15.97:0/3081610495 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f693009c220 con 0x7f693c0772f0 2024-04-26T21:48:23.315 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-26T21:48:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.317+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 msgr2=0x7f69240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.317+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 0x7f69240a6100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f692802a930 tx=0x7f6928000bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.317+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 msgr2=0x7f693c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.317+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f693000d770 tx=0x7f693000dc30 comp rx=0 tx=0).stop 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 shutdown_connections 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f693c078ed0 0x7f693c1a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f693c078530 0x7f693c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f69240a3c40 0x7f69240a6100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 --2- 172.21.15.97:0/3081610495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f693c0772f0 0x7f693c19f100 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 >> 172.21.15.97:0/3081610495 conn(0x7f693c100120 msgr2=0x7f693c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:23.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 shutdown_connections 2024-04-26T21:48:23.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.318+0000 7f6943600640 1 -- 172.21.15.97:0/3081610495 wait complete. 2024-04-26T21:48:23.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:48:23.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 -- 172.21.15.97:0/2685650768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353010a940 msgr2=0x7f35301111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 --2- 172.21.15.97:0/2685650768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353010a940 0x7f35301111d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3528009ca0 tx=0x7f352802a8b0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 -- 172.21.15.97:0/2685650768 shutdown_connections 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 --2- 172.21.15.97:0/2685650768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353010a940 0x7f35301111d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 --2- 172.21.15.97:0/2685650768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530109f80 0x7f353010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 --2- 172.21.15.97:0/2685650768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3530108d90 0x7f35301091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 -- 172.21.15.97:0/2685650768 >> 172.21.15.97:0/2685650768 conn(0x7f3530077d30 msgr2=0x7f353007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:23.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 -- 172.21.15.97:0/2685650768 shutdown_connections 2024-04-26T21:48:23.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.534+0000 7f3535600640 1 -- 172.21.15.97:0/2685650768 wait complete. 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.535+0000 7f3535600640 1 Processor -- start 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.535+0000 7f3535600640 1 -- start start 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 0x7f353019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353010a940 0x7f35301a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3530114770 con 0x7f3530109f80 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f35301145f0 con 0x7f353010a940 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3535600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35301148f0 con 0x7f3530108d90 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33096/0 (socket says 172.21.15.97:33096) 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 -- 172.21.15.97:0/1852181631 learned_addr learned my addr 172.21.15.97:0/1852181631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f352fe00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 0x7f353019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3534c00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353010a940 0x7f35301a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 -- 172.21.15.97:0/1852181631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 msgr2=0x7f353019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 0x7f353019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 -- 172.21.15.97:0/1852181631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353010a940 msgr2=0x7f35301a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353010a940 0x7f35301a6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f3527e00640 1 -- 172.21.15.97:0/1852181631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35301a6a70 con 0x7f3530109f80 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.536+0000 7f352fe00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 0x7f353019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f3527e00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f351800d770 tx=0x7f351800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518017070 con 0x7f3530109f80 2024-04-26T21:48:23.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35180054b0 con 0x7f3530109f80 2024-04-26T21:48:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35301a6d60 con 0x7f3530109f80 2024-04-26T21:48:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518004c60 con 0x7f3530109f80 2024-04-26T21:48:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.537+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301a7200 con 0x7f3530109f80 2024-04-26T21:48:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.538+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35301100d0 con 0x7f3530109f80 2024-04-26T21:48:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.540+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3518004e00 con 0x7f3530109f80 2024-04-26T21:48:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.542+0000 7f352d600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 0x7f35140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.542+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f35180d0080 con 0x7f3530109f80 2024-04-26T21:48:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.542+0000 7f352fe00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 0x7f35140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.543+0000 7f352fe00640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 0x7f35140a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3530109de0 tx=0x7f3520005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.546+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f351809c910 con 0x7f3530109f80 2024-04-26T21:48:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.758+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f353002dd40 con 0x7f3530109f80 2024-04-26T21:48:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.759+0000 7f352d600640 1 -- 172.21.15.97:0/1852181631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f351809c210 con 0x7f3530109f80 2024-04-26T21:48:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/117675142 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1403922799 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3008702290 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2383506754 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/2998338527 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1403922799 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3810497084 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2341580572 2024-04-27T21:21:20.482343+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3660896935 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/2998338527 2024-04-27T21:20:46.262192+0000 2024-04-26T21:48:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 msgr2=0x7f35140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 0x7f35140a6100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3530109de0 tx=0x7f3520005e70 comp rx=0 tx=0).stop 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 msgr2=0x7f353019f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f351800d770 tx=0x7f351800dc30 comp rx=0 tx=0).stop 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 shutdown_connections 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353010a940 0x7f35301a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f35140a3c40 0x7f35140a6100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3530109f80 0x7f353019f390 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 --2- 172.21.15.97:0/1852181631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3530108d90 0x7f353019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 >> 172.21.15.97:0/1852181631 conn(0x7f3530077d30 msgr2=0x7f353010d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 shutdown_connections 2024-04-26T21:48:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:23.762+0000 7f3535600640 1 -- 172.21.15.97:0/1852181631 wait complete. 2024-04-26T21:48:23.780 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-04-26T21:48:23.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:23.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:48:23.856 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[80356]: fuse finished with error 0 and tester_r 0 2024-04-26T21:48:23.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:48:24.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:23 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3081610495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:24.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:23 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1852181631' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:23 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3081610495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:24.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:23 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1852181631' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:23 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3081610495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:48:24.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:23 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1852181631' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:48:25.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:24 smithi111 ceph-mon[31001]: pgmap v891: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:25.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:24 smithi143 ceph-mon[31461]: pgmap v891: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:25.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:24 smithi097 ceph-mon[28127]: pgmap v891: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:48:27.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:26 smithi143 ceph-mon[31461]: pgmap v892: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:48:27.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:26 smithi097 ceph-mon[28127]: pgmap v892: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:48:27.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:26 smithi111 ceph-mon[31001]: pgmap v892: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-26T21:48:29.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:28 smithi143 ceph-mon[31461]: pgmap v893: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-26T21:48:29.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:28 smithi097 ceph-mon[28127]: pgmap v893: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-26T21:48:29.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:28 smithi111 ceph-mon[31001]: pgmap v893: 97 pgs: 97 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-26T21:48:29.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:48:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:29.964 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:48:30.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:30.034 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-26T21:48:30.087 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:48:30.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:30.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:48:30.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:48:30.176 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-26T21:48:30.297 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T21:48:30.307 INFO:tasks.cephadm:Teardown begin 2024-04-26T21:48:30.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:48:30.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 -- 172.21.15.97:0/2237705533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0101880 msgr2=0x7f12a0101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 --2- 172.21.15.97:0/2237705533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0101880 0x7f12a0101ca0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1290009ca0 tx=0x7f129002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 -- 172.21.15.97:0/2237705533 shutdown_connections 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 --2- 172.21.15.97:0/2237705533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0103430 0x7f12a0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 --2- 172.21.15.97:0/2237705533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 --2- 172.21.15.97:0/2237705533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0101880 0x7f12a0101ca0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 -- 172.21.15.97:0/2237705533 >> 172.21.15.97:0/2237705533 conn(0x7f12a00fcdf0 msgr2=0x7f12a00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 -- 172.21.15.97:0/2237705533 shutdown_connections 2024-04-26T21:48:30.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.487+0000 7f12a4e00640 1 -- 172.21.15.97:0/2237705533 wait complete. 2024-04-26T21:48:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.488+0000 7f12a4e00640 1 Processor -- start 2024-04-26T21:48:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.488+0000 7f12a4e00640 1 -- start start 2024-04-26T21:48:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0101880 0x7f12a010c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a010a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f12a010e3e0 con 0x7f12a0103430 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f12a010e260 con 0x7f12a0102a70 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f12a4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f12a010e560 con 0x7f12a0101880 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33110/0 (socket says 172.21.15.97:33110) 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 -- 172.21.15.97:0/1420773144 learned_addr learned my addr 172.21.15.97:0/1420773144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f1295400640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a010a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129de00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0101880 0x7f12a010c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 -- 172.21.15.97:0/1420773144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0101880 msgr2=0x7f12a010c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0101880 0x7f12a010c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 -- 172.21.15.97:0/1420773144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 msgr2=0x7f12a010a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a010a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f129e800640 1 -- 172.21.15.97:0/1420773144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a010b610 con 0x7f12a0103430 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.489+0000 7f1295400640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a010a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f129e800640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f129800c970 tx=0x7f129800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298007bb0 con 0x7f12a0103430 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01a2be0 con 0x7f12a0103430 2024-04-26T21:48:30.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1298007d50 con 0x7f12a0103430 2024-04-26T21:48:30.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298004d50 con 0x7f12a0103430 2024-04-26T21:48:30.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.490+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01a3130 con 0x7f12a0103430 2024-04-26T21:48:30.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.491+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a0108c70 con 0x7f12a0103430 2024-04-26T21:48:30.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.493+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1298005020 con 0x7f12a0103430 2024-04-26T21:48:30.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.495+0000 7f1297400640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 0x7f12840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.495+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f12980d0630 con 0x7f12a0103430 2024-04-26T21:48:30.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.495+0000 7f129de00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 0x7f12840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.496+0000 7f129de00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 0x7f12840a6100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1290009430 tx=0x7f1290000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.499+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1298014030 con 0x7f12a0103430 2024-04-26T21:48:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.741+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12a002dd20 con 0x7f12a0103430 2024-04-26T21:48:30.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.744+0000 7f1297400640 1 -- 172.21.15.97:0/1420773144 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11937 (secure 0 0 0) 0x7f129809cec0 con 0x7f12a0103430 2024-04-26T21:48:30.745 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:48:30.745 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:33:18.347998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34460,34502,34433,34400,24487]},"id":1}]} 2024-04-26T21:48:30.745 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-26T21:48:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 msgr2=0x7f12840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 0x7f12840a6100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1290009430 tx=0x7f1290000f40 comp rx=0 tx=0).stop 2024-04-26T21:48:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 msgr2=0x7f12a010ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f129800c970 tx=0x7f129800ce30 comp rx=0 tx=0).stop 2024-04-26T21:48:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 shutdown_connections 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f12840a3c40 0x7f12840a6100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12a0103430 0x7f12a010ad80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12a0102a70 0x7f12a010a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 --2- 172.21.15.97:0/1420773144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a0101880 0x7f12a010c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 >> 172.21.15.97:0/1420773144 conn(0x7f12a00fcdf0 msgr2=0x7f12a0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 shutdown_connections 2024-04-26T21:48:30.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.746+0000 7f12a4e00640 1 -- 172.21.15.97:0/1420773144 wait complete. 2024-04-26T21:48:30.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:48:30.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.972+0000 7f7e51400640 1 -- 172.21.15.97:0/2788838163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c104320 msgr2=0x7f7e4c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.972+0000 7f7e51400640 1 --2- 172.21.15.97:0/2788838163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c104320 0x7f7e4c10abb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009900 tx=0x7f7e4402a410 comp rx=0 tx=0).stop 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 -- 172.21.15.97:0/2788838163 shutdown_connections 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 --2- 172.21.15.97:0/2788838163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c104320 0x7f7e4c10abb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 --2- 172.21.15.97:0/2788838163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c103960 0x7f7e4c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 --2- 172.21.15.97:0/2788838163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 0x7f7e4c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 -- 172.21.15.97:0/2788838163 >> 172.21.15.97:0/2788838163 conn(0x7f7e4c0fdcc0 msgr2=0x7f7e4c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:30.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 -- 172.21.15.97:0/2788838163 shutdown_connections 2024-04-26T21:48:30.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.973+0000 7f7e51400640 1 -- 172.21.15.97:0/2788838163 wait complete. 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.974+0000 7f7e51400640 1 Processor -- start 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.974+0000 7f7e51400640 1 -- start start 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 0x7f7e4c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c104320 0x7f7e4c1a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10e150 con 0x7f7e4c103960 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10dfd0 con 0x7f7e4c102770 2024-04-26T21:48:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e51400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10e2d0 con 0x7f7e4c104320 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41582/0 (socket says 172.21.15.97:41582) 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e4b400640 1 -- 172.21.15.97:0/841300448 learned_addr learned my addr 172.21.15.97:0/841300448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e4be00640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 0x7f7e4c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.975+0000 7f7e50a00640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c104320 0x7f7e4c1a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 -- 172.21.15.97:0/841300448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c104320 msgr2=0x7f7e4c1a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c104320 0x7f7e4c1a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 -- 172.21.15.97:0/841300448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 msgr2=0x7f7e4c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:30.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 0x7f7e4c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 -- 172.21.15.97:0/841300448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c1a2680 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e4b400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d770 tx=0x7f7e3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34017070 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e340054b0 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c1a2970 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34004c60 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.976+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c1a2e70 con 0x7f7e4c103960 2024-04-26T21:48:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.978+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c1a30a0 con 0x7f7e4c103960 2024-04-26T21:48:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.980+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7e34004e00 con 0x7f7e4c103960 2024-04-26T21:48:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.982+0000 7f7e48c00640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 0x7f7e300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.982+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f7e340cffe0 con 0x7f7e4c103960 2024-04-26T21:48:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.982+0000 7f7e4be00640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 0x7f7e300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.983+0000 7f7e4be00640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 0x7f7e300a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c1037c0 tx=0x7f7e3c0098f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:30.989+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e3409c8b0 con 0x7f7e4c103960 2024-04-26T21:48:31.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:30 smithi097 ceph-mon[28127]: pgmap v894: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-26T21:48:31.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:30 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1420773144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:31.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.235+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e4c1a3270 con 0x7f7e4c103960 2024-04-26T21:48:31.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.237+0000 7f7e48c00640 1 -- 172.21.15.97:0/841300448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11937 (secure 0 0 0) 0x7f7e3409c1b0 con 0x7f7e4c103960 2024-04-26T21:48:31.238 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:48:31.238 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4203195871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4203195871},{"type":"v1","addr":"172.21.15.97:6835","nonce":4203195871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/847789147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":847789147},{"type":"v1","addr":"172.21.15.97:6837","nonce":847789147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:30:53.245300+0000","modified":"2024-04-26T21:33:18.347998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34502,"mds_1":34460,"mds_2":34433,"mds_3":34400,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/3674554302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3674554302},{"type":"v1","addr":"172.21.15.97:6839","nonce":3674554302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1900627986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1900627986},{"type":"v1","addr":"172.21.15.97:6841","nonce":1900627986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/845773802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":845773802},{"type":"v1","addr":"172.21.15.143:6833","nonce":845773802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/4086222297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4086222297},{"type":"v1","addr":"172.21.15.143:6835","nonce":4086222297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/1658655529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1658655529},{"type":"v1","addr":"172.21.15.143:6837","nonce":1658655529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.143:6839/2048200369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2048200369},{"type":"v1","addr":"172.21.15.143:6839","nonce":2048200369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.111:6833/4057011733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":4057011733},{"type":"v1","addr":"172.21.15.111:6833","nonce":4057011733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34433":{"gid":34433,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6835/738887742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":738887742},{"type":"v1","addr":"172.21.15.111:6835","nonce":738887742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6837/1944730061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1944730061},{"type":"v1","addr":"172.21.15.111:6837","nonce":1944730061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34502":{"gid":34502,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2198422773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2198422773},{"type":"v1","addr":"172.21.15.111:6839","nonce":2198422773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34502,"qdb_cluster":[34460,34502,34433,34400,24487]},"id":1}]} 2024-04-26T21:48:31.238 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-04-26T21:48:31.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 msgr2=0x7f7e300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 0x7f7e300a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c1037c0 tx=0x7f7e3c0098f0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 msgr2=0x7f7e4c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d770 tx=0x7f7e3400dc30 comp rx=0 tx=0).stop 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 shutdown_connections 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e4c104320 0x7f7e4c1a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f7e300a3c40 0x7f7e300a6100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4c103960 0x7f7e4c19afa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 --2- 172.21.15.97:0/841300448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7e4c102770 0x7f7e4c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 >> 172.21.15.97:0/841300448 conn(0x7f7e4c0fdcc0 msgr2=0x7f7e4c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 shutdown_connections 2024-04-26T21:48:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.240+0000 7f7e51400640 1 -- 172.21.15.97:0/841300448 wait complete. 2024-04-26T21:48:31.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:48:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:30 smithi143 ceph-mon[31461]: pgmap v894: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-26T21:48:31.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:30 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1420773144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:31.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:30 smithi111 ceph-mon[31001]: pgmap v894: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-26T21:48:31.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:30 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1420773144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 -- 172.21.15.97:0/2466332375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594105b50 msgr2=0x7f0594105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 --2- 172.21.15.97:0/2466332375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594105b50 0x7f0594105fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0580009900 tx=0x7f058002a400 comp rx=0 tx=0).stop 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 -- 172.21.15.97:0/2466332375 shutdown_connections 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 --2- 172.21.15.97:0/2466332375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594106510 0x7f059410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 --2- 172.21.15.97:0/2466332375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594105b50 0x7f0594105fd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 --2- 172.21.15.97:0/2466332375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594104960 0x7f0594104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 -- 172.21.15.97:0/2466332375 >> 172.21.15.97:0/2466332375 conn(0x7f05940ffeb0 msgr2=0x7f0594102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 -- 172.21.15.97:0/2466332375 shutdown_connections 2024-04-26T21:48:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.458+0000 7f0593200640 1 -- 172.21.15.97:0/2466332375 wait complete. 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.459+0000 7f0593200640 1 Processor -- start 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.459+0000 7f0593200640 1 -- start start 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594105b50 0x7f059419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 0x7f05941a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0594110320 con 0x7f0594104960 2024-04-26T21:48:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f05941101a0 con 0x7f0594105b50 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0593200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f05941104a0 con 0x7f0594106510 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0591e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0591e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41600/0 (socket says 172.21.15.97:41600) 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0591e00640 1 -- 172.21.15.97:0/731660282 learned_addr learned my addr 172.21.15.97:0/731660282 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0591400640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594105b50 0x7f059419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.460+0000 7f0592800640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 0x7f05941a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 -- 172.21.15.97:0/731660282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 msgr2=0x7f05941a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 0x7f05941a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 -- 172.21.15.97:0/731660282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594105b50 msgr2=0x7f059419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594105b50 0x7f059419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 -- 172.21.15.97:0/731660282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05941a2680 con 0x7f0594104960 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0592800640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 0x7f05941a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0591e00640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f058400c970 tx=0x7f058400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584007c10 con 0x7f0594104960 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0584007db0 con 0x7f0594104960 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058400f7d0 con 0x7f0594104960 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05941a2970 con 0x7f0594104960 2024-04-26T21:48:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.461+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05941a2e40 con 0x7f0594104960 2024-04-26T21:48:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.462+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0594069ac0 con 0x7f0594104960 2024-04-26T21:48:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.464+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0584018480 con 0x7f0594104960 2024-04-26T21:48:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.466+0000 7f058aa00640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 0x7f05780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.466+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f05840d09a0 con 0x7f0594104960 2024-04-26T21:48:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.466+0000 7f0591400640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 0x7f05780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.467+0000 7f0591400640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 0x7f05780a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0580000c00 tx=0x7f0580031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.470+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f058409d2c0 con 0x7f0594104960 2024-04-26T21:48:31.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.675+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f059402db90 con 0x7f0594104960 2024-04-26T21:48:31.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.679+0000 7f058aa00640 1 -- 172.21.15.97:0/731660282 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22532 (secure 0 0 0) 0x7f058409cbc0 con 0x7f0594104960 2024-04-26T21:48:31.680 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T21:48:31.680 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"9ce681f8-0412-11ef-bc93-c7b262605968","created":"2024-04-26T21:20:14.183633+0000","modified":"2024-04-26T21:31:21.811271+0000","last_up_change":"2024-04-26T21:29:47.482564+0000","last_in_change":"2024-04-26T21:29:21.539387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:24:36.675748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:30:51.224775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:30:52.223998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b100e5ac-c305-4e0b-80d1-246e9107f082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6803","nonce":3890416768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6805","nonce":3890416768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6809","nonce":3890416768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3890416768},{"type":"v1","addr":"172.21.15.97:6807","nonce":3890416768}]},"public_addr":"172.21.15.97:6803/3890416768","cluster_addr":"172.21.15.97:6805/3890416768","heartbeat_back_addr":"172.21.15.97:6809/3890416768","heartbeat_front_addr":"172.21.15.97:6807/3890416768","state":["exists","up"]},{"osd":1,"uuid":"2702d8b6-9f28-421d-b2d1-9c38a9900bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6801","nonce":4100839926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6803","nonce":4100839926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6807","nonce":4100839926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":4100839926},{"type":"v1","addr":"172.21.15.111:6805","nonce":4100839926}]},"public_addr":"172.21.15.111:6801/4100839926","cluster_addr":"172.21.15.111:6803/4100839926","heartbeat_back_addr":"172.21.15.111:6807/4100839926","heartbeat_front_addr":"172.21.15.111:6805/4100839926","state":["exists","up"]},{"osd":2,"uuid":"7059e770-39f5-4850-bfa3-521f33c394d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6801","nonce":1100983067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6803","nonce":1100983067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6807","nonce":1100983067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1100983067},{"type":"v1","addr":"172.21.15.143:6805","nonce":1100983067}]},"public_addr":"172.21.15.143:6801/1100983067","cluster_addr":"172.21.15.143:6803/1100983067","heartbeat_back_addr":"172.21.15.143:6807/1100983067","heartbeat_front_addr":"172.21.15.143:6805/1100983067","state":["exists","up"]},{"osd":3,"uuid":"5e8d302e-f0d1-4f1d-9464-905160e2eb4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6811","nonce":92668618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6813","nonce":92668618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6817","nonce":92668618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":92668618},{"type":"v1","addr":"172.21.15.97:6815","nonce":92668618}]},"public_addr":"172.21.15.97:6811/92668618","cluster_addr":"172.21.15.97:6813/92668618","heartbeat_back_addr":"172.21.15.97:6817/92668618","heartbeat_front_addr":"172.21.15.97:6815/92668618","state":["exists","up"]},{"osd":4,"uuid":"672a72b1-edd4-4392-b5ed-1167ae4c4205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6809","nonce":3555238271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6811","nonce":3555238271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6815","nonce":3555238271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3555238271},{"type":"v1","addr":"172.21.15.111:6813","nonce":3555238271}]},"public_addr":"172.21.15.111:6809/3555238271","cluster_addr":"172.21.15.111:6811/3555238271","heartbeat_back_addr":"172.21.15.111:6815/3555238271","heartbeat_front_addr":"172.21.15.111:6813/3555238271","state":["exists","up"]},{"osd":5,"uuid":"f52a82b9-6a36-4c0a-8287-f853589258c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6809","nonce":1354252557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6811","nonce":1354252557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6815","nonce":1354252557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1354252557},{"type":"v1","addr":"172.21.15.143:6813","nonce":1354252557}]},"public_addr":"172.21.15.143:6809/1354252557","cluster_addr":"172.21.15.143:6811/1354252557","heartbeat_back_addr":"172.21.15.143:6815/1354252557","heartbeat_front_addr":"172.21.15.143:6813/1354252557","state":["exists","up"]},{"osd":6,"uuid":"b3c68893-2e7a-4b3a-bbce-eb87ba907a25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6819","nonce":3234116941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6821","nonce":3234116941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6825","nonce":3234116941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3234116941},{"type":"v1","addr":"172.21.15.97:6823","nonce":3234116941}]},"public_addr":"172.21.15.97:6819/3234116941","cluster_addr":"172.21.15.97:6821/3234116941","heartbeat_back_addr":"172.21.15.97:6825/3234116941","heartbeat_front_addr":"172.21.15.97:6823/3234116941","state":["exists","up"]},{"osd":7,"uuid":"fb8c32d3-48dc-4e01-8e3b-127389176afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6817","nonce":3525126129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6819","nonce":3525126129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6823","nonce":3525126129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3525126129},{"type":"v1","addr":"172.21.15.111:6821","nonce":3525126129}]},"public_addr":"172.21.15.111:6817/3525126129","cluster_addr":"172.21.15.111:6819/3525126129","heartbeat_back_addr":"172.21.15.111:6823/3525126129","heartbeat_front_addr":"172.21.15.111:6821/3525126129","state":["exists","up"]},{"osd":8,"uuid":"5bd31888-d587-4094-a159-3c0f46dd71d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6817","nonce":2919334001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6819","nonce":2919334001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6823","nonce":2919334001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2919334001},{"type":"v1","addr":"172.21.15.143:6821","nonce":2919334001}]},"public_addr":"172.21.15.143:6817/2919334001","cluster_addr":"172.21.15.143:6819/2919334001","heartbeat_back_addr":"172.21.15.143:6823/2919334001","heartbeat_front_addr":"172.21.15.143:6821/2919334001","state":["exists","up"]},{"osd":9,"uuid":"ff1702b9-db9d-48f5-835f-4ababb3bc2ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6827","nonce":3567005259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6829","nonce":3567005259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6833","nonce":3567005259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3567005259},{"type":"v1","addr":"172.21.15.97:6831","nonce":3567005259}]},"public_addr":"172.21.15.97:6827/3567005259","cluster_addr":"172.21.15.97:6829/3567005259","heartbeat_back_addr":"172.21.15.97:6833/3567005259","heartbeat_front_addr":"172.21.15.97:6831/3567005259","state":["exists","up"]},{"osd":10,"uuid":"db65ed02-3eb7-4c51-a807-bf051e380b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6825","nonce":3431248583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6827","nonce":3431248583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6831","nonce":3431248583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3431248583},{"type":"v1","addr":"172.21.15.111:6829","nonce":3431248583}]},"public_addr":"172.21.15.111:6825/3431248583","cluster_addr":"172.21.15.111:6827/3431248583","heartbeat_back_addr":"172.21.15.111:6831/3431248583","heartbeat_front_addr":"172.21.15.111:6829/3431248583","state":["exists","up"]},{"osd":11,"uuid":"17cd4c20-11c9-426a-9504-09d4fe119a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6825","nonce":921154932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6827","nonce":921154932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6831","nonce":921154932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":921154932},{"type":"v1","addr":"172.21.15.143:6829","nonce":921154932}]},"public_addr":"172.21.15.143:6825/921154932","cluster_addr":"172.21.15.143:6827/921154932","heartbeat_back_addr":"172.21.15.143:6831/921154932","heartbeat_front_addr":"172.21.15.143:6829/921154932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:25.227708+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:23:58.873409+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:24:33.102260+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:06.596248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:25:40.802026+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:15.001235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:26:50.009322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:23.476240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:27:58.549951+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:28:33.372959+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:09.334773+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:29:44.636013+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/117675142":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6801/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3008702290":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2383506754":"2024-04-27T21:21:20.482343+0000","172.21.15.97:6800/2998338527":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6800/1403922799":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3810497084":"2024-04-27T21:20:46.262192+0000","172.21.15.97:0/2341580572":"2024-04-27T21:21:20.482343+0000","172.21.15.97:0/3660896935":"2024-04-27T21:20:46.262192+0000","172.21.15.97:6801/2998338527":"2024-04-27T21:20:46.262192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:48:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 msgr2=0x7f05780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 0x7f05780a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0580000c00 tx=0x7f0580031400 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 msgr2=0x7f059419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f058400c970 tx=0x7f058400ce30 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 shutdown_connections 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0594106510 0x7f05941a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0594105b50 0x7f059419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f05780a3c40 0x7f05780a6100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 --2- 172.21.15.97:0/731660282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0594104960 0x7f059419aa60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 >> 172.21.15.97:0/731660282 conn(0x7f05940ffeb0 msgr2=0x7f0594108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 shutdown_connections 2024-04-26T21:48:31.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.682+0000 7f0593200640 1 -- 172.21.15.97:0/731660282 wait complete. 2024-04-26T21:48:31.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:48:31.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 -- 172.21.15.97:0/1321697744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c109f80 msgr2=0x7f104c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 --2- 172.21.15.97:0/1321697744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c109f80 0x7f104c10a400 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1034009900 tx=0x7f103402a400 comp rx=0 tx=0).stop 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 -- 172.21.15.97:0/1321697744 shutdown_connections 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 --2- 172.21.15.97:0/1321697744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c10a940 0x7f104c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 --2- 172.21.15.97:0/1321697744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c109f80 0x7f104c10a400 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 --2- 172.21.15.97:0/1321697744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c108d90 0x7f104c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 -- 172.21.15.97:0/1321697744 >> 172.21.15.97:0/1321697744 conn(0x7f104c077d30 msgr2=0x7f104c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 -- 172.21.15.97:0/1321697744 shutdown_connections 2024-04-26T21:48:31.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.905+0000 7f1050a00640 1 -- 172.21.15.97:0/1321697744 wait complete. 2024-04-26T21:48:31.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.906+0000 7f1050a00640 1 Processor -- start 2024-04-26T21:48:31.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.906+0000 7f1050a00640 1 -- start start 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.906+0000 7f1050a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c108d90 0x7f104c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.906+0000 7f1050a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c109f80 0x7f104c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f1050a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f1050a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1147a0 con 0x7f104c10a940 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f1050a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f104c114620 con 0x7f104c108d90 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f1050a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f104c114920 con 0x7f104c109f80 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41626/0 (socket says 172.21.15.97:41626) 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 -- 172.21.15.97:0/2503829820 learned_addr learned my addr 172.21.15.97:0/2503829820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104a800640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c108d90 0x7f104c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f1043e00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c109f80 0x7f104c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 -- 172.21.15.97:0/2503829820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c109f80 msgr2=0x7f104c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c109f80 0x7f104c19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 -- 172.21.15.97:0/2503829820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c108d90 msgr2=0x7f104c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c108d90 0x7f104c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.907+0000 7f104b200640 1 -- 172.21.15.97:0/2503829820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c1a69d0 con 0x7f104c10a940 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f104b200640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f104400cce0 tx=0x7f104400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104400f450 con 0x7f104c10a940 2024-04-26T21:48:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f104400fa80 con 0x7f104c10a940 2024-04-26T21:48:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c1a6cc0 con 0x7f104c10a940 2024-04-26T21:48:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044018ad0 con 0x7f104c10a940 2024-04-26T21:48:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.908+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1a71c0 con 0x7f104c10a940 2024-04-26T21:48:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.909+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c110050 con 0x7f104c10a940 2024-04-26T21:48:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.911+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f104400f5f0 con 0x7f104c10a940 2024-04-26T21:48:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.913+0000 7f1043400640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 0x7f10300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.913+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8132+0+0 (secure 0 0 0) 0x7f10440d0d90 con 0x7f104c10a940 2024-04-26T21:48:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.913+0000 7f104a800640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 0x7f10300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.914+0000 7f104a800640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 0x7f10300a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f103c0058c0 tx=0x7f103c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:31.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:31.917+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1044014030 con 0x7f104c10a940 2024-04-26T21:48:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.163+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f104c02dd20 con 0x7f104c10a940 2024-04-26T21:48:32.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.168+0000 7f1043400640 1 -- 172.21.15.97:0/2503829820 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f104409d620 con 0x7f104c10a940 2024-04-26T21:48:32.169 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:48:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.172+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 msgr2=0x7f10300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.172+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 0x7f10300a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f103c0058c0 tx=0x7f103c005850 comp rx=0 tx=0).stop 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.172+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 msgr2=0x7f104c1a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.172+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f104400cce0 tx=0x7f104400a620 comp rx=0 tx=0).stop 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 shutdown_connections 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f10300a3cc0 0x7f10300a6180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f104c10a940 0x7f104c1a6360 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f104c109f80 0x7f104c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 --2- 172.21.15.97:0/2503829820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f104c108d90 0x7f104c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 >> 172.21.15.97:0/2503829820 conn(0x7f104c077d30 msgr2=0x7f104c10d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 shutdown_connections 2024-04-26T21:48:32.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.173+0000 7f1050a00640 1 -- 172.21.15.97:0/2503829820 wait complete. 2024-04-26T21:48:32.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:48:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 -- 172.21.15.97:0/594514242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0106740 msgr2=0x7f32c010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 --2- 172.21.15.97:0/594514242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0106740 0x7f32c010cfd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 -- 172.21.15.97:0/594514242 shutdown_connections 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 --2- 172.21.15.97:0/594514242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0106740 0x7f32c010cfd0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 --2- 172.21.15.97:0/594514242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 0x7f32c0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 --2- 172.21.15.97:0/594514242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0104b90 0x7f32c0104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 -- 172.21.15.97:0/594514242 >> 172.21.15.97:0/594514242 conn(0x7f32c0100120 msgr2=0x7f32c0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:32.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 -- 172.21.15.97:0/594514242 shutdown_connections 2024-04-26T21:48:32.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.379+0000 7f32c6e00640 1 -- 172.21.15.97:0/594514242 wait complete. 2024-04-26T21:48:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.380+0000 7f32c6e00640 1 Processor -- start 2024-04-26T21:48:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.380+0000 7f32c6e00640 1 -- start start 2024-04-26T21:48:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 0x7f32c019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 0x7f32c01a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01105a0 con 0x7f32c0104b90 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0110420 con 0x7f32c0105d80 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0110720 con 0x7f32c0106740 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41678/0 (socket says 172.21.15.97:41678) 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 -- 172.21.15.97:0/1187723322 learned_addr learned my addr 172.21.15.97:0/1187723322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bf400640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 0x7f32c019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32c4e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 0x7f32c01a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 -- 172.21.15.97:0/1187723322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 msgr2=0x7f32c01a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 0x7f32c01a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 -- 172.21.15.97:0/1187723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 msgr2=0x7f32c019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 0x7f32c019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.381+0000 7f32bfe00640 1 -- 172.21.15.97:0/1187723322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a27a0 con 0x7f32c0104b90 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32c4e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 0x7f32c01a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32bfe00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c970 tx=0x7f32b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8007c10 con 0x7f32c0104b90 2024-04-26T21:48:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b8007db0 con 0x7f32c0104b90 2024-04-26T21:48:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01a2a90 con 0x7f32c0104b90 2024-04-26T21:48:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800f7d0 con 0x7f32c0104b90 2024-04-26T21:48:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.382+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a2fe0 con 0x7f32c0104b90 2024-04-26T21:48:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.383+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c010beb0 con 0x7f32c0104b90 2024-04-26T21:48:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.385+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f32b8018480 con 0x7f32c0104b90 2024-04-26T21:48:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.387+0000 7f32bcc00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 0x7f32a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.388+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8562+0+0 (secure 0 0 0) 0x7f32b80d0ac0 con 0x7f32c0104b90 2024-04-26T21:48:32.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.388+0000 7f32bf400640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 0x7f32a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.388+0000 7f32bf400640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 0x7f32a40a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f32c019c090 tx=0x7f32a80075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:32.391 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mon[31001]: pgmap v895: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-26T21:48:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/841300448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/731660282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.392 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.392 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.392 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mds[60939]: -- 172.21.15.111:0/2442068062 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e2fbb7a40 con 0x559e2fb8b000 2024-04-26T21:48:32.393 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: starting mds.h at 2024-04-26T21:48:32.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mds[61256]: -- 172.21.15.111:0/1131414895 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7b1571a40 con 0x55c7b1544800 2024-04-26T21:48:32.393 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: starting mds.b at 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mds[60638]: -- 172.21.15.111:0/4154794704 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9d6eb5a40 con 0x55c9d6e89000 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: starting mds.e at 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.394 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.395 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-mds[60326]: -- 172.21.15.111:0/1345703217 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca36f7da40 con 0x55ca36f50800 2024-04-26T21:48:32.395 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:48:32 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: starting mds.k at 2024-04-26T21:48:32.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.392+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f32b809d220 con 0x7f32c0104b90 2024-04-26T21:48:32.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-mon[31461]: pgmap v895: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-26T21:48:32.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/841300448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/731660282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: starting mds.i at 2024-04-26T21:48:32.546 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.546 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.546 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-mds[61484]: -- 172.21.15.143:0/529083430 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ba6407a40 con 0x558ba63db000 2024-04-26T21:48:32.546 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: starting mds.f at 2024-04-26T21:48:32.546 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-mds[61783]: -- 172.21.15.143:0/4160854234 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8d33f1a40 con 0x55a8d33c4c00 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: starting mds.c at 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.547 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:48:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: starting mds.l at 2024-04-26T21:48:32.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-mon[28127]: pgmap v895: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-26T21:48:32.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/841300448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/731660282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a[28122]: 2024-04-26T21:48:32.165+0000 7f6679000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-mds[69826]: -- 172.21.15.97:0/3714112617 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e31baba40 con 0x556e31b7ec00 2024-04-26T21:48:32.564 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: starting mds.a at 2024-04-26T21:48:32.565 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: ignoring --setuser ceph since I am not root 2024-04-26T21:48:32.565 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: ignoring --setgroup ceph since I am not root 2024-04-26T21:48:32.565 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-mds[69459]: -- 172.21.15.97:0/1220318561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2cf675a40 con 0x55c2cf648800 2024-04-26T21:48:32.565 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:48:32 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: starting mds.g at 2024-04-26T21:48:32.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:32.616+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f32c002dad0 con 0x7f32c0104b90 2024-04-26T21:48:33.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.174+0000 7f32bcc00640 1 -- 172.21.15.97:0/1187723322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f32c002dad0 con 0x7f32c0104b90 2024-04-26T21:48:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.178+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 msgr2=0x7f32a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.178+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 0x7f32a40a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f32c019c090 tx=0x7f32a80075b0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.178+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 msgr2=0x7f32c019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.178+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c970 tx=0x7f32b800ce30 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 shutdown_connections 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32c0106740 0x7f32c01a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32c0105d80 0x7f32c019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f32a40a3c40 0x7f32a40a6100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 --2- 172.21.15.97:0/1187723322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c0104b90 0x7f32c019ab80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 >> 172.21.15.97:0/1187723322 conn(0x7f32c0100120 msgr2=0x7f32c0109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 shutdown_connections 2024-04-26T21:48:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.179+0000 7f32c6e00640 1 -- 172.21.15.97:0/1187723322 wait complete. 2024-04-26T21:48:33.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:48:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:48:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:48:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:48:33.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:48:33.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:48:33.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:48:33.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:33 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 -- 172.21.15.97:0/2580597794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 msgr2=0x7f9d9c1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 --2- 172.21.15.97:0/2580597794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c1060c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009900 tx=0x7f9d8802a410 comp rx=0 tx=0).stop 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 -- 172.21.15.97:0/2580597794 shutdown_connections 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 --2- 172.21.15.97:0/2580597794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c106600 0x7f9d9c10ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 --2- 172.21.15.97:0/2580597794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c1060c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 --2- 172.21.15.97:0/2580597794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c104a50 0x7f9d9c104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 -- 172.21.15.97:0/2580597794 >> 172.21.15.97:0/2580597794 conn(0x7f9d9c0fffe0 msgr2=0x7f9d9c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:33.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 -- 172.21.15.97:0/2580597794 shutdown_connections 2024-04-26T21:48:33.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.406+0000 7f9d9be00640 1 -- 172.21.15.97:0/2580597794 wait complete. 2024-04-26T21:48:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.407+0000 7f9d9be00640 1 Processor -- start 2024-04-26T21:48:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.407+0000 7f9d9be00640 1 -- start start 2024-04-26T21:48:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c104a50 0x7f9d9c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c106600 0x7f9d9c1a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110410 con 0x7f9d9c105c40 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110290 con 0x7f9d9c106600 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110590 con 0x7f9d9c104a50 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41690/0 (socket says 172.21.15.97:41690) 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9a000640 1 -- 172.21.15.97:0/1548074903 learned_addr learned my addr 172.21.15.97:0/1548074903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9b400640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c106600 0x7f9d9c1a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.408+0000 7f9d9aa00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c104a50 0x7f9d9c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 -- 172.21.15.97:0/1548074903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c104a50 msgr2=0x7f9d9c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c104a50 0x7f9d9c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 -- 172.21.15.97:0/1548074903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c106600 msgr2=0x7f9d9c1a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c106600 0x7f9d9c1a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 -- 172.21.15.97:0/1548074903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c1a27a0 con 0x7f9d9c105c40 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9a000640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009230 tx=0x7f9d880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8803f030 con 0x7f9d9c105c40 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c1a2a30 con 0x7f9d9c105c40 2024-04-26T21:48:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d88003c90 con 0x7f9d9c105c40 2024-04-26T21:48:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88035070 con 0x7f9d9c105c40 2024-04-26T21:48:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.409+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a2f20 con 0x7f9d9c105c40 2024-04-26T21:48:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.411+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c10bee0 con 0x7f9d9c105c40 2024-04-26T21:48:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.412+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d88042030 con 0x7f9d9c105c40 2024-04-26T21:48:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.414+0000 7f9d93400640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.414+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8562+0+0 (secure 0 0 0) 0x7f9d880f0f00 con 0x7f9d9c105c40 2024-04-26T21:48:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.414+0000 7f9d9aa00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.415+0000 7f9d9aa00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 0x7f9d800a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c005fd0 tx=0x7f9d8c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.418+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9d880bd630 con 0x7f9d9c105c40 2024-04-26T21:48:33.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:48:33.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:33 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:48:33.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/2503829820' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:48:33.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:33 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:48:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:33.622+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9d9c02dad0 con 0x7f9d9c105c40 2024-04-26T21:48:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.177+0000 7f9d93400640 1 -- 172.21.15.97:0/1548074903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f9d880bcf30 con 0x7f9d9c105c40 2024-04-26T21:48:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:48:34.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.181+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 msgr2=0x7f9d800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:34.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.181+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 0x7f9d800a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c005fd0 tx=0x7f9d8c005850 comp rx=0 tx=0).stop 2024-04-26T21:48:34.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 msgr2=0x7f9d9c19b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:34.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009230 tx=0x7f9d880042f0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 shutdown_connections 2024-04-26T21:48:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d9c106600 0x7f9d9c1a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9c105c40 0x7f9d9c19b0c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1548074903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9c104a50 0x7f9d9c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.182+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 >> 172.21.15.97:0/1548074903 conn(0x7f9d9c0fffe0 msgr2=0x7f9d9c1090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.183+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 shutdown_connections 2024-04-26T21:48:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.183+0000 7f9d9be00640 1 -- 172.21.15.97:0/1548074903 wait complete. 2024-04-26T21:48:34.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: pgmap v897: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: Cluster is now healthy 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:48:34.393 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6838/1223333774,v1:172.21.15.97:6839/1223333774] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.97:6840/2000064008,v1:172.21.15.97:6841/2000064008] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6832/1414073249,v1:172.21.15.111:6833/1414073249] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6834/3790024754,v1:172.21.15.111:6835/3790024754] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6836/176418159,v1:172.21.15.111:6837/176418159] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6832/31629859,v1:172.21.15.143:6836/31629859] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6833/255326827,v1:172.21.15.143:6837/255326827] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.111:6838/4259528313,v1:172.21.15.111:6839/4259528313] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6834/3288654202,v1:172.21.15.143:6838/3288654202] up:boot 2024-04-26T21:48:34.394 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: mds.? [v2:172.21.15.143:6835/2831945745,v1:172.21.15.143:6839/2831945745] up:boot 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: fsmap 12 up:standby 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:48:34.395 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:48:34.396 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:48:34.396 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:48:34.396 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:34 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 -- 172.21.15.97:0/340701386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128078530 msgr2=0x7f0128078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 --2- 172.21.15.97:0/340701386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128078530 0x7f0128078990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0114009900 tx=0x7f011402a410 comp rx=0 tx=0).stop 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 -- 172.21.15.97:0/340701386 shutdown_connections 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 --2- 172.21.15.97:0/340701386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0128078ed0 0x7f0128079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 --2- 172.21.15.97:0/340701386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128078530 0x7f0128078990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 --2- 172.21.15.97:0/340701386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01280772f0 0x7f0128077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 -- 172.21.15.97:0/340701386 >> 172.21.15.97:0/340701386 conn(0x7f01281000e0 msgr2=0x7f0128102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 -- 172.21.15.97:0/340701386 shutdown_connections 2024-04-26T21:48:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.400+0000 7f012ce00640 1 -- 172.21.15.97:0/340701386 wait complete. 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.401+0000 7f012ce00640 1 Processor -- start 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.401+0000 7f012ce00640 1 -- start start 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0128078530 0x7f012819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0128078ed0 0x7f01281a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f01281149c0 con 0x7f01280772f0 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0128114840 con 0x7f0128078ed0 2024-04-26T21:48:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f012ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0128114b40 con 0x7f0128078530 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41722/0 (socket says 172.21.15.97:41722) 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 -- 172.21.15.97:0/3785579391 learned_addr learned my addr 172.21.15.97:0/3785579391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0126800640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0128078ed0 0x7f01281a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 -- 172.21.15.97:0/3785579391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0128078530 msgr2=0x7f012819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0128078530 0x7f012819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 -- 172.21.15.97:0/3785579391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0128078ed0 msgr2=0x7f01281a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0128078ed0 0x7f01281a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.402+0000 7f0125e00640 1 -- 172.21.15.97:0/3785579391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01281a6ca0 con 0x7f01280772f0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f0125e00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f011800c920 tx=0x7f011800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118007c70 con 0x7f01280772f0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0118007e10 con 0x7f01280772f0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118018710 con 0x7f01280772f0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01281a6f90 con 0x7f01280772f0 2024-04-26T21:48:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.403+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281a7450 con 0x7f01280772f0 2024-04-26T21:48:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.404+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0128111710 con 0x7f01280772f0 2024-04-26T21:48:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.406+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f011800f450 con 0x7f01280772f0 2024-04-26T21:48:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.407+0000 7f011ea00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 0x7f010c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.408+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8058+0+0 (secure 0 0 0) 0x7f01180d0760 con 0x7f01280772f0 2024-04-26T21:48:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.408+0000 7f0125400640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 0x7f010c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.409+0000 7f0125400640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 0x7f010c0a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0114009230 tx=0x7f0114000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:34.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.412+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0118014030 con 0x7f01280772f0 2024-04-26T21:48:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: pgmap v897: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:48:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:48:34.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: Cluster is now healthy 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6838/1223333774,v1:172.21.15.97:6839/1223333774] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.97:6840/2000064008,v1:172.21.15.97:6841/2000064008] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6832/1414073249,v1:172.21.15.111:6833/1414073249] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6834/3790024754,v1:172.21.15.111:6835/3790024754] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6836/176418159,v1:172.21.15.111:6837/176418159] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6832/31629859,v1:172.21.15.143:6836/31629859] up:boot 2024-04-26T21:48:34.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6833/255326827,v1:172.21.15.143:6837/255326827] up:boot 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.111:6838/4259528313,v1:172.21.15.111:6839/4259528313] up:boot 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6834/3288654202,v1:172.21.15.143:6838/3288654202] up:boot 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: mds.? [v2:172.21.15.143:6835/2831945745,v1:172.21.15.143:6839/2831945745] up:boot 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: fsmap 12 up:standby 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:48:34.547 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:48:34.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:48:34.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:48:34.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:48:34.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:48:34.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:34 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:34.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: pgmap v897: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:48:34.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: Cluster is now healthy 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1187723322' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6838/1223333774,v1:172.21.15.97:6839/1223333774] up:boot 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.97:6840/2000064008,v1:172.21.15.97:6841/2000064008] up:boot 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6832/1414073249,v1:172.21.15.111:6833/1414073249] up:boot 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6834/3790024754,v1:172.21.15.111:6835/3790024754] up:boot 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6836/176418159,v1:172.21.15.111:6837/176418159] up:boot 2024-04-26T21:48:34.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6832/31629859,v1:172.21.15.143:6836/31629859] up:boot 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6833/255326827,v1:172.21.15.143:6837/255326827] up:boot 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.111:6838/4259528313,v1:172.21.15.111:6839/4259528313] up:boot 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6834/3288654202,v1:172.21.15.143:6838/3288654202] up:boot 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: mds.? [v2:172.21.15.143:6835/2831945745,v1:172.21.15.143:6839/2831945745] up:boot 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: fsmap 12 up:standby 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:48:34.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='mgr.14152 172.21.15.97:0/1711266405' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:48:34.567 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:34 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:34.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:34.623+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f012802dad0 con 0x7f01280772f0 2024-04-26T21:48:35.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.181+0000 7f011ea00640 1 -- 172.21.15.97:0/3785579391 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7f011800de20 con 0x7f01280772f0 2024-04-26T21:48:35.182 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-26T21:48:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.185+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 msgr2=0x7f010c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.185+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 0x7f010c0a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0114009230 tx=0x7f0114000bc0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.185+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 msgr2=0x7f012819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.185+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f011800c920 tx=0x7f011800cde0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.187+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 shutdown_connections 2024-04-26T21:48:35.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.187+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0128078ed0 0x7f01281a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.187+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0128078530 0x7f012819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.187+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:6800/3172837618,v1:172.21.15.97:6801/3172837618] conn(0x7f010c0a3c40 0x7f010c0a6100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.187+0000 7f012ce00640 1 --2- 172.21.15.97:0/3785579391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01280772f0 0x7f012819f080 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:35.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.188+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 >> 172.21.15.97:0/3785579391 conn(0x7f01281000e0 msgr2=0x7f01281018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:35.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.188+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 shutdown_connections 2024-04-26T21:48:35.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T21:48:35.188+0000 7f012ce00640 1 -- 172.21.15.97:0/3785579391 wait complete. 2024-04-26T21:48:35.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:48:35.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:48:35.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:48:35.453 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:35 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:35.453 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:35 smithi143 ceph-mon[31461]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:48:35.454 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:35 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:35.468 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T21:48:35.468 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:48:35.526 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:48:35.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:35.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:35 smithi097 ceph-mon[28127]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:48:35.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:35 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:35.582 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:48:35.639 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T21:48:35.639 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T21:48:35.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:48:35.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:35 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/1548074903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:35.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:35 smithi111 ceph-mon[31001]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:48:35.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:35 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:48:36.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:35 smithi097 systemd[1]: Stopping Ceph mon.a for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:36.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:36 smithi143 ceph-mon[31461]: pgmap v899: 65 pgs: 65 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:48:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:36 smithi143 ceph-mon[31461]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:36.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:36 smithi143 ceph-mon[31461]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:48:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 ceph-mon[28127]: pgmap v899: 65 pgs: 65 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:48:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 ceph-mon[28127]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:36.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 ceph-mon[28127]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:48:36.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a[28122]: 2024-04-26T21:48:36.203+0000 7f667fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:36.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a[28122]: 2024-04-26T21:48:36.203+0000 7f667fe00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T21:48:36.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 podman[84315]: 2024-04-26 21:48:36.296926952 +0000 UTC m=+0.218738631 container died 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:48:36.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:36 smithi111 ceph-mon[31001]: pgmap v899: 65 pgs: 65 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:48:36.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:36 smithi111 ceph-mon[31001]: from='client.? 172.21.15.97:0/3785579391' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:48:36.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:36 smithi111 ceph-mon[31001]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:48:37.063 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 podman[84315]: 2024-04-26 21:48:36.854328487 +0000 UTC m=+0.776140176 container cleanup 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T21:48:37.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:36 smithi097 bash[84315]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:48:37.521 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 26 21:48:37 smithi097 podman[84329]: 2024-04-26 21:48:37.077599483 +0000 UTC m=+0.763897865 container remove 8132cccb75ffec9f32ba0c99089a3abd6e499dbf5edc7806164f8db31378ffe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:48:37.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T21:48:37.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:37.683 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T21:48:37.683 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T21:48:37.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.b 2024-04-26T21:48:38.142 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:37 smithi111 systemd[1]: Stopping Ceph mon.b for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:38.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:38 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b[30996]: 2024-04-26T21:48:38.281+0000 7f2556200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:38.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:38 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b[30996]: 2024-04-26T21:48:38.281+0000 7f2556200700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:48:38.643 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:38 smithi111 podman[65654]: 2024-04-26 21:48:38.435994008 +0000 UTC m=+0.296196781 container died 7ebaf616a25df178b61edf2f921e0e6f8989cdd445e7cc9266afe45feb34dd9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T21:48:39.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:38 smithi111 podman[65654]: 2024-04-26 21:48:38.99823107 +0000 UTC m=+0.858433897 container cleanup 7ebaf616a25df178b61edf2f921e0e6f8989cdd445e7cc9266afe45feb34dd9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:48:39.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:38 smithi111 bash[65654]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b 2024-04-26T21:48:39.392 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 26 21:48:39 smithi111 podman[65669]: 2024-04-26 21:48:39.257965205 +0000 UTC m=+0.799079465 container remove 7ebaf616a25df178b61edf2f921e0e6f8989cdd445e7cc9266afe45feb34dd9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:48:39.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T21:48:39.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:39.938 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T21:48:39.938 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T21:48:39.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.c 2024-04-26T21:48:40.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:40 smithi143 systemd[1]: Stopping Ceph mon.c for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:40.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:40 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c[31455]: 2024-04-26T21:48:40.484+0000 7f4027a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:40.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:40 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c[31455]: 2024-04-26T21:48:40.484+0000 7f4027a00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T21:48:40.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:40 smithi143 podman[66511]: 2024-04-26 21:48:40.610465474 +0000 UTC m=+0.244266203 container died 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:48:41.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:41 smithi143 podman[66511]: 2024-04-26 21:48:41.212132101 +0000 UTC m=+0.845932842 container cleanup 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-26T21:48:41.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:41 smithi143 bash[66511]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c 2024-04-26T21:48:41.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:Apr 26 21:48:41 smithi143 podman[66525]: 2024-04-26 21:48:41.438999138 +0000 UTC m=+0.813506332 container remove 9148503ed2f403522c77cdbcea931663dac4ef38c04e48e42e35109a4db07008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T21:48:41.887 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T21:48:42.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:42.029 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T21:48:42.030 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T21:48:42.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:48:42.563 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:42 smithi097 systemd[1]: Stopping Ceph mgr.x for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:42.981 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:48:42.589+0000 7f3b0d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:42.981 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x[28368]: 2024-04-26T21:48:42.589+0000 7f3b0d400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T21:48:42.981 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:42 smithi097 podman[84440]: 2024-04-26 21:48:42.685805061 +0000 UTC m=+0.247022773 container died 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:48:43.313 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:43 smithi097 podman[84440]: 2024-04-26 21:48:43.183757317 +0000 UTC m=+0.744975047 container cleanup 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T21:48:43.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:43 smithi097 bash[84440]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x 2024-04-26T21:48:43.813 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 26 21:48:43 smithi097 podman[84454]: 2024-04-26 21:48:43.385158178 +0000 UTC m=+0.692122107 container remove 75825ae1845a13240882743b5ceaf5e2b2cb315def696b86c7097cadf0f41aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T21:48:43.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T21:48:43.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:44.157 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T21:48:44.157 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T21:48:44.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.y 2024-04-26T21:48:44.642 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:44 smithi111 systemd[1]: Stopping Ceph mgr.y for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:45.119 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:44 smithi111 podman[65776]: 2024-04-26 21:48:44.783830369 +0000 UTC m=+0.211079431 container died 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:48:45.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:45 smithi111 podman[65776]: 2024-04-26 21:48:45.270031609 +0000 UTC m=+0.697280698 container cleanup 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T21:48:45.392 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:45 smithi111 bash[65776]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y 2024-04-26T21:48:45.892 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:45 smithi111 podman[65790]: 2024-04-26 21:48:45.547174267 +0000 UTC m=+0.750740979 container remove 4fd235dbe8211dc0f2dcf98e53018dbec2eeafe50c290c2e5be0729f7f171407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T21:48:45.892 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 26 21:48:45 smithi111 systemd[1]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-26T21:48:45.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T21:48:46.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:46.088 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T21:48:46.088 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T21:48:46.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.z 2024-04-26T21:48:46.544 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:46 smithi143 systemd[1]: Stopping Ceph mgr.z for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:47.130 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:46 smithi143 podman[66650]: 2024-04-26 21:48:46.808403586 +0000 UTC m=+0.284020805 container died 53d47b99af6e52dfc67be35a94f15064fa1ff2a7217e276757bfa15d704c3331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:48:47.542 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:47 smithi143 podman[66650]: 2024-04-26 21:48:47.283025056 +0000 UTC m=+0.758642245 container cleanup 53d47b99af6e52dfc67be35a94f15064fa1ff2a7217e276757bfa15d704c3331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:48:47.543 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:47 smithi143 bash[66650]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z 2024-04-26T21:48:47.794 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:47 smithi143 podman[66664]: 2024-04-26 21:48:47.542284569 +0000 UTC m=+0.719970078 container remove 53d47b99af6e52dfc67be35a94f15064fa1ff2a7217e276757bfa15d704c3331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T21:48:47.794 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Apr 26 21:48:47 smithi143 systemd[1]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-26T21:48:48.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T21:48:48.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:48.169 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T21:48:48.169 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T21:48:48.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.0 2024-04-26T21:48:48.563 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:48 smithi097 systemd[1]: Stopping Ceph osd.0 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:49.064 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:48 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0[43242]: 2024-04-26T21:48:48.718+0000 7fb980400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:49.064 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:48 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0[43242]: 2024-04-26T21:48:48.718+0000 7fb980400700 -1 osd.0 93 *** Got signal Terminated *** 2024-04-26T21:48:49.064 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:48 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0[43242]: 2024-04-26T21:48:48.718+0000 7fb980400700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:48:54.410 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:54 smithi097 podman[84560]: 2024-04-26 21:48:54.133148584 +0000 UTC m=+5.555132545 container died c21fc1dfc29ef42491524801be08fd4d8f0ff785cd4996235488da114936f89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T21:48:54.714 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:54 smithi097 podman[84560]: 2024-04-26 21:48:54.512929487 +0000 UTC m=+5.934913475 container cleanup c21fc1dfc29ef42491524801be08fd4d8f0ff785cd4996235488da114936f89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T21:48:54.715 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:54 smithi097 bash[84560]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0 2024-04-26T21:48:55.063 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:54 smithi097 podman[84580]: 2024-04-26 21:48:54.713884818 +0000 UTC m=+0.564007611 container remove c21fc1dfc29ef42491524801be08fd4d8f0ff785cd4996235488da114936f89f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:48:55.694 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:55 smithi097 podman[84669]: 2024-04-26 21:48:55.439770921 +0000 UTC m=+0.124955982 container create daeef99f088bf9010a124511d4629b89f9140b6c10adff681323031e6fad5714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T21:48:55.694 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:55 smithi097 podman[84669]: 2024-04-26 21:48:55.350293154 +0000 UTC m=+0.035478184 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:48:55.694 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:55 smithi097 podman[84669]: 2024-04-26 21:48:55.693172167 +0000 UTC m=+0.378357240 container init daeef99f088bf9010a124511d4629b89f9140b6c10adff681323031e6fad5714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:48:56.063 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:55 smithi097 podman[84669]: 2024-04-26 21:48:55.703258587 +0000 UTC m=+0.388443649 container start daeef99f088bf9010a124511d4629b89f9140b6c10adff681323031e6fad5714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:48:56.064 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:55 smithi097 podman[84669]: 2024-04-26 21:48:55.783484959 +0000 UTC m=+0.468670061 container attach daeef99f088bf9010a124511d4629b89f9140b6c10adff681323031e6fad5714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:48:56.541 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 26 21:48:56 smithi097 podman[84669]: 2024-04-26 21:48:56.146935649 +0000 UTC m=+0.832120722 container died daeef99f088bf9010a124511d4629b89f9140b6c10adff681323031e6fad5714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:48:56.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T21:48:57.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:48:57.070 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T21:48:57.071 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T21:48:57.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.1 2024-04-26T21:48:57.641 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:48:57 smithi111 systemd[1]: Stopping Ceph osd.1 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:48:57.892 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:48:57 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1[35583]: 2024-04-26T21:48:57.638+0000 7f5f40800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:48:57.892 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:48:57 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1[35583]: 2024-04-26T21:48:57.639+0000 7f5f40800700 -1 osd.1 93 *** Got signal Terminated *** 2024-04-26T21:48:57.892 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:48:57 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1[35583]: 2024-04-26T21:48:57.639+0000 7f5f40800700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:03.329 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:03 smithi111 podman[65910]: 2024-04-26 21:49:03.037508157 +0000 UTC m=+5.525830111 container died 5c66ff95ea0e6af4a604267500e0108ffbd2c2d0a21e40d7d727f54854b7adf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T21:49:03.641 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:03 smithi111 podman[65910]: 2024-04-26 21:49:03.442124035 +0000 UTC m=+5.930446025 container cleanup 5c66ff95ea0e6af4a604267500e0108ffbd2c2d0a21e40d7d727f54854b7adf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T21:49:03.642 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:03 smithi111 bash[65910]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1 2024-04-26T21:49:04.142 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:03 smithi111 podman[65928]: 2024-04-26 21:49:03.751786312 +0000 UTC m=+0.695877526 container remove 5c66ff95ea0e6af4a604267500e0108ffbd2c2d0a21e40d7d727f54854b7adf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:49:04.892 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:04 smithi111 podman[66012]: 2024-04-26 21:49:04.52423268 +0000 UTC m=+0.039171150 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:04.892 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:04 smithi111 podman[66012]: 2024-04-26 21:49:04.654779518 +0000 UTC m=+0.169717972 container create be22d1f5145f029f95bbc44f1b1e6d131d5085a05e8fdfaa955baf06683850bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-04-26T21:49:05.393 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:04 smithi111 podman[66012]: 2024-04-26 21:49:04.967785827 +0000 UTC m=+0.482724327 container init be22d1f5145f029f95bbc44f1b1e6d131d5085a05e8fdfaa955baf06683850bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:49:05.393 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:04 smithi111 podman[66012]: 2024-04-26 21:49:04.979373118 +0000 UTC m=+0.494311635 container start be22d1f5145f029f95bbc44f1b1e6d131d5085a05e8fdfaa955baf06683850bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:49:05.393 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:05 smithi111 podman[66012]: 2024-04-26 21:49:05.108442076 +0000 UTC m=+0.623380590 container attach be22d1f5145f029f95bbc44f1b1e6d131d5085a05e8fdfaa955baf06683850bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T21:49:05.774 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 26 21:49:05 smithi111 podman[66012]: 2024-04-26 21:49:05.436578903 +0000 UTC m=+0.951517366 container died be22d1f5145f029f95bbc44f1b1e6d131d5085a05e8fdfaa955baf06683850bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:49:06.147 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T21:49:06.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:06.251 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T21:49:06.251 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T21:49:06.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.2 2024-04-26T21:49:06.794 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:06 smithi143 systemd[1]: Stopping Ceph osd.2 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:07.295 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2[36643]: 2024-04-26T21:49:06.834+0000 7f4ae8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:07.295 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2[36643]: 2024-04-26T21:49:06.834+0000 7f4ae8a00700 -1 osd.2 93 *** Got signal Terminated *** 2024-04-26T21:49:07.295 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2[36643]: 2024-04-26T21:49:06.834+0000 7f4ae8a00700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:12.724 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:12 smithi143 podman[66773]: 2024-04-26 21:49:12.375300415 +0000 UTC m=+5.661203126 container died 6da82a3b014e73b9791651a444465c0b7c116755cc79d34c4cf2ac79277206a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T21:49:13.044 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:12 smithi143 podman[66773]: 2024-04-26 21:49:12.902293503 +0000 UTC m=+6.188196229 container cleanup 6da82a3b014e73b9791651a444465c0b7c116755cc79d34c4cf2ac79277206a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:49:13.045 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:12 smithi143 bash[66773]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2 2024-04-26T21:49:13.544 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:13 smithi143 podman[66791]: 2024-04-26 21:49:13.120040336 +0000 UTC m=+0.729676633 container remove 6da82a3b014e73b9791651a444465c0b7c116755cc79d34c4cf2ac79277206a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T21:49:14.274 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:13 smithi143 podman[66878]: 2024-04-26 21:49:13.866320074 +0000 UTC m=+0.036421956 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:14.275 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:14 smithi143 podman[66878]: 2024-04-26 21:49:14.007515075 +0000 UTC m=+0.177616993 container create c910faefcc82a8c0f633377e41eba6edafa534edd22978922f635fb149bd9c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:49:14.544 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:14 smithi143 podman[66878]: 2024-04-26 21:49:14.274147132 +0000 UTC m=+0.444249069 container init c910faefcc82a8c0f633377e41eba6edafa534edd22978922f635fb149bd9c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T21:49:14.544 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:14 smithi143 podman[66878]: 2024-04-26 21:49:14.285925111 +0000 UTC m=+0.456027023 container start c910faefcc82a8c0f633377e41eba6edafa534edd22978922f635fb149bd9c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:49:14.545 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:14 smithi143 podman[66878]: 2024-04-26 21:49:14.364844887 +0000 UTC m=+0.534946808 container attach c910faefcc82a8c0f633377e41eba6edafa534edd22978922f635fb149bd9c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T21:49:15.044 INFO:journalctl@ceph.osd.2.smithi143.stdout:Apr 26 21:49:14 smithi143 podman[66878]: 2024-04-26 21:49:14.736582068 +0000 UTC m=+0.906683964 container died c910faefcc82a8c0f633377e41eba6edafa534edd22978922f635fb149bd9c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:49:15.532 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T21:49:15.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:15.626 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T21:49:15.627 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T21:49:15.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.3 2024-04-26T21:49:16.063 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:15 smithi097 systemd[1]: Stopping Ceph osd.3 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:16.564 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:16 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3[50008]: 2024-04-26T21:49:16.204+0000 7f3b4a800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:16.565 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:16 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3[50008]: 2024-04-26T21:49:16.204+0000 7f3b4a800700 -1 osd.3 93 *** Got signal Terminated *** 2024-04-26T21:49:16.565 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:16 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3[50008]: 2024-04-26T21:49:16.204+0000 7f3b4a800700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:21.903 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:21 smithi097 podman[84789]: 2024-04-26 21:49:21.623175214 +0000 UTC m=+5.540622894 container died 182f5f469f2aaef33bbdec7fee32029acaf6577fe8d9a85373208653e1ac54de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T21:49:22.313 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:21 smithi097 podman[84789]: 2024-04-26 21:49:21.997050214 +0000 UTC m=+5.914497950 container cleanup 182f5f469f2aaef33bbdec7fee32029acaf6577fe8d9a85373208653e1ac54de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T21:49:22.313 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:21 smithi097 bash[84789]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3 2024-04-26T21:49:22.813 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:22 smithi097 podman[84806]: 2024-04-26 21:49:22.314835316 +0000 UTC m=+0.679224109 container remove 182f5f469f2aaef33bbdec7fee32029acaf6577fe8d9a85373208653e1ac54de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:49:23.212 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:22 smithi097 podman[84892]: 2024-04-26 21:49:22.841745504 +0000 UTC m=+0.037757968 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:23.212 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:22 smithi097 podman[84892]: 2024-04-26 21:49:22.940912948 +0000 UTC m=+0.136925433 container create 19315e54379d4e5feafc704c5b2dc82ebfb6ba44a2f5bcbb9d1f2022669745e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:49:23.212 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:23 smithi097 podman[84892]: 2024-04-26 21:49:23.211342834 +0000 UTC m=+0.407355325 container init 19315e54379d4e5feafc704c5b2dc82ebfb6ba44a2f5bcbb9d1f2022669745e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:49:23.467 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:23 smithi097 podman[84892]: 2024-04-26 21:49:23.221831229 +0000 UTC m=+0.417843744 container start 19315e54379d4e5feafc704c5b2dc82ebfb6ba44a2f5bcbb9d1f2022669745e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:49:23.468 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:23 smithi097 podman[84892]: 2024-04-26 21:49:23.309771922 +0000 UTC m=+0.505784470 container attach 19315e54379d4e5feafc704c5b2dc82ebfb6ba44a2f5bcbb9d1f2022669745e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:49:23.774 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 26 21:49:23 smithi097 podman[84892]: 2024-04-26 21:49:23.680264683 +0000 UTC m=+0.876277168 container died 19315e54379d4e5feafc704c5b2dc82ebfb6ba44a2f5bcbb9d1f2022669745e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:49:24.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T21:49:24.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:24.406 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T21:49:24.407 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T21:49:24.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.4 2024-04-26T21:49:24.892 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:24 smithi111 systemd[1]: Stopping Ceph osd.4 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:25.259 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4[42231]: 2024-04-26T21:49:24.948+0000 7f8e55400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:25.259 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4[42231]: 2024-04-26T21:49:24.948+0000 7f8e55400700 -1 osd.4 93 *** Got signal Terminated *** 2024-04-26T21:49:25.259 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4[42231]: 2024-04-26T21:49:24.948+0000 7f8e55400700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:30.642 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:30 smithi111 podman[66122]: 2024-04-26 21:49:30.387065338 +0000 UTC m=+5.565122290 container died 6cfe20ba6000c32be1306522f28535340c3dbbea04f523a82803d5ec21f361f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:49:31.101 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:30 smithi111 podman[66122]: 2024-04-26 21:49:30.790845931 +0000 UTC m=+5.968902908 container cleanup 6cfe20ba6000c32be1306522f28535340c3dbbea04f523a82803d5ec21f361f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:49:31.101 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:30 smithi111 bash[66122]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4 2024-04-26T21:49:31.392 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:31 smithi111 podman[66153]: 2024-04-26 21:49:31.100405608 +0000 UTC m=+0.701434958 container remove 6cfe20ba6000c32be1306522f28535340c3dbbea04f523a82803d5ec21f361f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:49:32.277 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:31.926238212 +0000 UTC m=+0.032023449 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:32.277 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:32.051457668 +0000 UTC m=+0.157242902 container create 5b0efdf3fc2d6a97890fd23359176b0b834098a91eea666e2158208a0393b096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T21:49:32.641 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:32.363759232 +0000 UTC m=+0.469544550 container init 5b0efdf3fc2d6a97890fd23359176b0b834098a91eea666e2158208a0393b096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T21:49:32.642 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:32.375585602 +0000 UTC m=+0.481370846 container start 5b0efdf3fc2d6a97890fd23359176b0b834098a91eea666e2158208a0393b096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:49:32.642 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:32.454081471 +0000 UTC m=+0.559866772 container attach 5b0efdf3fc2d6a97890fd23359176b0b834098a91eea666e2158208a0393b096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:49:33.137 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 26 21:49:32 smithi111 podman[66240]: 2024-04-26 21:49:32.82594078 +0000 UTC m=+0.931726025 container died 5b0efdf3fc2d6a97890fd23359176b0b834098a91eea666e2158208a0393b096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:49:33.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T21:49:33.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:33.672 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T21:49:33.672 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T21:49:33.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.5 2024-04-26T21:49:34.044 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:33 smithi143 systemd[1]: Stopping Ceph osd.5 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:34.544 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:34 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5[43391]: 2024-04-26T21:49:34.278+0000 7f3a86a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:34.545 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:34 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5[43391]: 2024-04-26T21:49:34.278+0000 7f3a86a00700 -1 osd.5 93 *** Got signal Terminated *** 2024-04-26T21:49:34.545 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:34 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5[43391]: 2024-04-26T21:49:34.278+0000 7f3a86a00700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:40.008 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:39 smithi143 podman[66996]: 2024-04-26 21:49:39.685561819 +0000 UTC m=+5.556029859 container died 416a9b639720c2b2b3d70f7e15df4e783c09b008ac0174c37917f9f2c9c9c5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:49:40.294 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:40 smithi143 podman[66996]: 2024-04-26 21:49:40.127315732 +0000 UTC m=+5.997783809 container cleanup 416a9b639720c2b2b3d70f7e15df4e783c09b008ac0174c37917f9f2c9c9c5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:49:40.295 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:40 smithi143 bash[66996]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5 2024-04-26T21:49:40.794 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:40 smithi143 podman[67015]: 2024-04-26 21:49:40.412120382 +0000 UTC m=+0.717516883 container remove 416a9b639720c2b2b3d70f7e15df4e783c09b008ac0174c37917f9f2c9c9c5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:49:41.545 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:41 smithi143 podman[67105]: 2024-04-26 21:49:41.171753251 +0000 UTC m=+0.033324333 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:41.545 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:41 smithi143 podman[67105]: 2024-04-26 21:49:41.287839768 +0000 UTC m=+0.149410883 container create 599e9d90bdb57d50861173ddd4262e90feadf27e35be45e756e39dcd83f98908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:49:42.041 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:41 smithi143 podman[67105]: 2024-04-26 21:49:41.574616531 +0000 UTC m=+0.436187653 container init 599e9d90bdb57d50861173ddd4262e90feadf27e35be45e756e39dcd83f98908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T21:49:42.041 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:41 smithi143 podman[67105]: 2024-04-26 21:49:41.588600378 +0000 UTC m=+0.450171500 container start 599e9d90bdb57d50861173ddd4262e90feadf27e35be45e756e39dcd83f98908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T21:49:42.041 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:41 smithi143 podman[67105]: 2024-04-26 21:49:41.673174119 +0000 UTC m=+0.534745264 container attach 599e9d90bdb57d50861173ddd4262e90feadf27e35be45e756e39dcd83f98908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:49:42.294 INFO:journalctl@ceph.osd.5.smithi143.stdout:Apr 26 21:49:42 smithi143 podman[67105]: 2024-04-26 21:49:42.04114852 +0000 UTC m=+0.902719641 container died 599e9d90bdb57d50861173ddd4262e90feadf27e35be45e756e39dcd83f98908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T21:49:42.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T21:49:42.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:42.907 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T21:49:42.907 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T21:49:42.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.6 2024-04-26T21:49:43.313 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:43 smithi097 systemd[1]: Stopping Ceph osd.6 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:43.813 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6[56626]: 2024-04-26T21:49:43.454+0000 7f6db4a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:43.814 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6[56626]: 2024-04-26T21:49:43.454+0000 7f6db4a00700 -1 osd.6 93 *** Got signal Terminated *** 2024-04-26T21:49:43.814 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:43 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6[56626]: 2024-04-26T21:49:43.454+0000 7f6db4a00700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:49.296 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:49 smithi097 podman[85014]: 2024-04-26 21:49:49.007918609 +0000 UTC m=+5.700532668 container died f783cb106ab331194aa0eb99fb7defa6aaa4eef910e7b616e27dee0060f8f151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T21:49:49.563 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:49 smithi097 podman[85014]: 2024-04-26 21:49:49.423050955 +0000 UTC m=+6.115665053 container cleanup f783cb106ab331194aa0eb99fb7defa6aaa4eef910e7b616e27dee0060f8f151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:49:49.564 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:49 smithi097 bash[85014]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6 2024-04-26T21:49:50.063 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:49 smithi097 podman[85035]: 2024-04-26 21:49:49.640748338 +0000 UTC m=+0.623000964 container remove f783cb106ab331194aa0eb99fb7defa6aaa4eef910e7b616e27dee0060f8f151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:49:50.771 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:50 smithi097 podman[85125]: 2024-04-26 21:49:50.383381768 +0000 UTC m=+0.037735986 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:50.771 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:50 smithi097 podman[85125]: 2024-04-26 21:49:50.499941649 +0000 UTC m=+0.154295896 container create df68f90d5c95ca7a5a653a3e2d6be720b1b063cb70bade1cdb2740474955df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T21:49:50.773 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:50 smithi097 podman[85125]: 2024-04-26 21:49:50.770737115 +0000 UTC m=+0.425091368 container init df68f90d5c95ca7a5a653a3e2d6be720b1b063cb70bade1cdb2740474955df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T21:49:51.063 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:50 smithi097 podman[85125]: 2024-04-26 21:49:50.783676792 +0000 UTC m=+0.438031046 container start df68f90d5c95ca7a5a653a3e2d6be720b1b063cb70bade1cdb2740474955df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T21:49:51.063 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:50 smithi097 podman[85125]: 2024-04-26 21:49:50.860393252 +0000 UTC m=+0.514747545 container attach df68f90d5c95ca7a5a653a3e2d6be720b1b063cb70bade1cdb2740474955df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:49:51.564 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 26 21:49:51 smithi097 podman[85125]: 2024-04-26 21:49:51.244286576 +0000 UTC m=+0.898640788 container died df68f90d5c95ca7a5a653a3e2d6be720b1b063cb70bade1cdb2740474955df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:49:51.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T21:49:52.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:49:52.069 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T21:49:52.070 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T21:49:52.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.7 2024-04-26T21:49:52.633 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:52 smithi111 systemd[1]: Stopping Ceph osd.7 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:49:52.892 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:52 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7[49312]: 2024-04-26T21:49:52.631+0000 7f5641000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:49:52.892 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:52 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7[49312]: 2024-04-26T21:49:52.631+0000 7f5641000700 -1 osd.7 93 *** Got signal Terminated *** 2024-04-26T21:49:52.892 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:52 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7[49312]: 2024-04-26T21:49:52.632+0000 7f5641000700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:49:56.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:49:55 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:49:55.579+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:49:56.392 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:49:56 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:49:56.103+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:49:57.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:49:56 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:49:56.566+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:49:57.391 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:49:57 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:49:57.138+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:49:58.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:49:57 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:49:57.564+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:49:58.063 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:57 smithi111 podman[66345]: 2024-04-26 21:49:57.992877518 +0000 UTC m=+5.511248027 container died 035cae164ee80cbe5e02d110db6efa548be48548db4e1be61cff024a94eb0208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:49:58.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:49:57 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:49:57.599+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:49:58.315 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:49:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:49:58.144+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:49:58.693 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:58 smithi111 podman[66345]: 2024-04-26 21:49:58.417440663 +0000 UTC m=+5.935811202 container cleanup 035cae164ee80cbe5e02d110db6efa548be48548db4e1be61cff024a94eb0208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T21:49:58.693 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:58 smithi111 bash[66345]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7 2024-04-26T21:49:58.974 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:58 smithi111 podman[66367]: 2024-04-26 21:49:58.693282891 +0000 UTC m=+0.686510212 container remove 035cae164ee80cbe5e02d110db6efa548be48548db4e1be61cff024a94eb0208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T21:49:59.044 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:49:58 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:49:58.568+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:49:59.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:49:58 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:49:58.587+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:49:59.392 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:49:59 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:49:59.150+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:49:59.716 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:59 smithi111 podman[66460]: 2024-04-26 21:49:59.322862108 +0000 UTC m=+0.031404604 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:49:59.716 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:59 smithi111 podman[66460]: 2024-04-26 21:49:59.427719877 +0000 UTC m=+0.136262368 container create db9be5ae8dcd5156ec053f7ea1aa43f11af0c3507aa0d944e035f555dbd4db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T21:49:59.716 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:59 smithi111 podman[66460]: 2024-04-26 21:49:59.715315463 +0000 UTC m=+0.423858034 container init db9be5ae8dcd5156ec053f7ea1aa43f11af0c3507aa0d944e035f555dbd4db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:49:59.813 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:49:59 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:49:59.561+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:00.044 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:49:59 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:49:59.566+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:00.134 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:59 smithi111 podman[66460]: 2024-04-26 21:49:59.726400209 +0000 UTC m=+0.434942743 container start db9be5ae8dcd5156ec053f7ea1aa43f11af0c3507aa0d944e035f555dbd4db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T21:50:00.135 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:49:59 smithi111 podman[66460]: 2024-04-26 21:49:59.846847236 +0000 UTC m=+0.555389774 container attach db9be5ae8dcd5156ec053f7ea1aa43f11af0c3507aa0d944e035f555dbd4db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:00.392 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:00 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:00.133+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:00.393 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 26 21:50:00 smithi111 podman[66460]: 2024-04-26 21:50:00.176345228 +0000 UTC m=+0.884887799 container died db9be5ae8dcd5156ec053f7ea1aa43f11af0c3507aa0d944e035f555dbd4db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T21:50:00.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:00 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:00.524+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:00.977 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T21:50:01.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:00 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:00.581+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:01.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:01.083 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T21:50:01.083 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T21:50:01.083 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.8 2024-04-26T21:50:01.391 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:01 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:01.169+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:01.500 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:01 smithi143 systemd[1]: Stopping Ceph osd.8 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:01.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:01.500+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:01.795 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:50:01.618+0000 7f3b76800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:01.795 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:50:01.618+0000 7f3b76800700 -1 osd.8 93 *** Got signal Terminated *** 2024-04-26T21:50:01.795 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:01 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:50:01.618+0000 7f3b76800700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:50:01.813 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:01 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:01.551+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:02.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:02.208+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:02.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:02 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:02.511+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:02.813 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:02 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:02.558+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:03.641 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:03 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:03.215+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:03.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:03 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:03.483+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:04.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:03 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:03.601+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:04.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:04 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:04.205+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:04.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:04 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:04.489+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:04.965 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:04 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:04.609+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:05.641 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:05 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:05.253+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:05.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:05 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:05.450+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:05.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:05 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:05.450+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:05.992 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:05 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:05.605+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:06.544 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8[49903]: 2024-04-26T21:50:06.260+0000 7f3b6d000700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.605989+0000 front 2024-04-26T21:49:02.606105+0000 (oldest deadline 2024-04-26T21:50:06.105710+0000) 2024-04-26T21:50:06.544 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:06.433+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:06.545 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:06 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:06.433+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:06.641 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:06.285+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:06.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:06.285+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:07.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:06 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:06.628+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:07.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:06 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:06.628+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:07.394 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:07 smithi143 podman[67226]: 2024-04-26 21:50:07.095292411 +0000 UTC m=+5.592676954 container died 7d5e210e733f300f5d58952b0d673a8f683a6f4486c793837f9dd781cf142893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T21:50:07.641 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:07 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:07.300+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:07.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:07 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:07.300+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:07.698 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:07 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:07.392+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:07.698 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:07 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:07.392+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:07.813 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:07 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:07.674+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:07.813 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:07 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:07.674+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:07.986 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:07 smithi143 podman[67226]: 2024-04-26 21:50:07.69816338 +0000 UTC m=+6.195547980 container cleanup 7d5e210e733f300f5d58952b0d673a8f683a6f4486c793837f9dd781cf142893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T21:50:07.986 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:07 smithi143 bash[67226]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8 2024-04-26T21:50:08.294 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:07 smithi143 podman[67244]: 2024-04-26 21:50:07.986060502 +0000 UTC m=+0.872578583 container remove 7d5e210e733f300f5d58952b0d673a8f683a6f4486c793837f9dd781cf142893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:50:08.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:08 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:08.340+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:08.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:08 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:08.340+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:08.739 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:08 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:08.406+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:08.739 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:08 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:08.406+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:08.739 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:08 smithi143 podman[67333]: 2024-04-26 21:50:08.454479733 +0000 UTC m=+0.034633523 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:08.739 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:08 smithi143 podman[67333]: 2024-04-26 21:50:08.5870116 +0000 UTC m=+0.167165415 container create c895ffea13c1afce2e3eb13cebe76dd649247c54409b90042cbda5e4486332e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T21:50:09.044 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:08 smithi143 podman[67333]: 2024-04-26 21:50:08.824035714 +0000 UTC m=+0.404189532 container init c895ffea13c1afce2e3eb13cebe76dd649247c54409b90042cbda5e4486332e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T21:50:09.045 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:08 smithi143 podman[67333]: 2024-04-26 21:50:08.836248934 +0000 UTC m=+0.416402725 container start c895ffea13c1afce2e3eb13cebe76dd649247c54409b90042cbda5e4486332e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:09.045 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:08 smithi143 podman[67333]: 2024-04-26 21:50:08.930835078 +0000 UTC m=+0.510988936 container attach c895ffea13c1afce2e3eb13cebe76dd649247c54409b90042cbda5e4486332e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:09.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:08 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:08.654+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:09.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:08 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:08.654+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:09.544 INFO:journalctl@ceph.osd.8.smithi143.stdout:Apr 26 21:50:09 smithi143 podman[67333]: 2024-04-26 21:50:09.274080679 +0000 UTC m=+0.854234474 container died c895ffea13c1afce2e3eb13cebe76dd649247c54409b90042cbda5e4486332e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:09.545 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:09 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:09.448+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:09.545 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:09 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:09.448+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:09.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:09 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:09.370+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:09.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:09 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:09.370+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:10.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:09 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:09.608+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:10.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:09 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:09.608+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:10.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T21:50:10.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:10.205 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T21:50:10.205 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T21:50:10.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.9 2024-04-26T21:50:10.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 systemd[1]: Stopping Ceph osd.9 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:10.641 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:10 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:10.356+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:10.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:10 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:10.356+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:10.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:10 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:10.489+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:10.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:10 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:10.489+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:11.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:10.571+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:11.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:10.571+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:11.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:10.797+0000 7f363fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:11.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:10.797+0000 7f363fc00700 -1 osd.9 93 *** Got signal Terminated *** 2024-04-26T21:50:11.065 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:10 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:10.797+0000 7f363fc00700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:50:11.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:11 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:11.376+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:11.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:11 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:11.376+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:11.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:11 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:11.536+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:11.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:11 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:11.536+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:12.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:11 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:11.606+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:12.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:11 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:11.606+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:12.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:12 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:12.533+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:12.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:12 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:12.533+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:12.891 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:12 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:12.426+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:12.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:12 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:12.426+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:13.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:12.616+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:13.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:12 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:12.616+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:13.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:13 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:13.533+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:13.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:13 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:13.533+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:13.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:13 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:13.454+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:13.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:13 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:13.454+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:13.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:13 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:13.454+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:14.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:13 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:13.628+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:14.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:13 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:13.628+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:14.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:13 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:13.628+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:07.094286+0000 front 2024-04-26T21:49:07.094394+0000 (oldest deadline 2024-04-26T21:50:12.994322+0000) 2024-04-26T21:50:14.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:14 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:14.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:14.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:14 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:14.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:14.891 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:14 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:14.480+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:14.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:14 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:14.480+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:14.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:14 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:14.480+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:15.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:14.620+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:15.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:14.620+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:15.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:14 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:14.620+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:07.094286+0000 front 2024-04-26T21:49:07.094394+0000 (oldest deadline 2024-04-26T21:50:12.994322+0000) 2024-04-26T21:50:15.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:15 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:15.531+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:15.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:15 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:15.531+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:15.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:15 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:15.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:15.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:15 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:15.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:15.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:15 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:15.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:16.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:15.610+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:51.387313+0000 front 2024-04-26T21:48:51.387874+0000 (oldest deadline 2024-04-26T21:49:54.887567+0000) 2024-04-26T21:50:16.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:15.610+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:01.891291+0000 front 2024-04-26T21:49:01.890942+0000 (oldest deadline 2024-04-26T21:50:05.991238+0000) 2024-04-26T21:50:16.064 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:15 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9[63213]: 2024-04-26T21:50:15.610+0000 7f363b400700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:07.094286+0000 front 2024-04-26T21:49:07.094394+0000 (oldest deadline 2024-04-26T21:50:12.994322+0000) 2024-04-26T21:50:16.503 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:16 smithi097 podman[85243]: 2024-04-26 21:50:16.224820685 +0000 UTC m=+5.556668801 container died 5eeb2ef29d05ad95cd194180b0e3180dc8cfe43b2ddd6724f53b62efc505cffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:16.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:16 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:16.522+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:16.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:16 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:16.522+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:16.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:16 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:16.522+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:16.807 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:16 smithi097 podman[85243]: 2024-04-26 21:50:16.623579611 +0000 UTC m=+5.955427770 container cleanup 5eeb2ef29d05ad95cd194180b0e3180dc8cfe43b2ddd6724f53b62efc505cffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:16.808 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:16 smithi097 bash[85243]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9 2024-04-26T21:50:16.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:16 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:16.435+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:16.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:16 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:16.435+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:16.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:16 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:16.435+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:17.063 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:16 smithi097 podman[85262]: 2024-04-26 21:50:16.807578253 +0000 UTC m=+0.567653767 container remove 5eeb2ef29d05ad95cd194180b0e3180dc8cfe43b2ddd6724f53b62efc505cffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T21:50:17.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:17 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:17.486+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:17.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:17 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:17.486+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:17.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:17 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:17.486+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:17.871 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:17 smithi097 podman[85350]: 2024-04-26 21:50:17.499656864 +0000 UTC m=+0.034862832 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:17.872 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:17 smithi097 podman[85350]: 2024-04-26 21:50:17.608503075 +0000 UTC m=+0.143709077 container create ae299a03b97693afbbe64d06a626c7a1f51dbb392d657467ae5f5a89dddaaab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:17.872 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:17 smithi097 podman[85350]: 2024-04-26 21:50:17.870840315 +0000 UTC m=+0.406046318 container init ae299a03b97693afbbe64d06a626c7a1f51dbb392d657467ae5f5a89dddaaab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T21:50:17.891 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:17 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:17.398+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:17.891 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:17 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:17.398+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:17.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:17 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:17.398+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:18.314 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:17 smithi097 podman[85350]: 2024-04-26 21:50:17.883238176 +0000 UTC m=+0.418444202 container start ae299a03b97693afbbe64d06a626c7a1f51dbb392d657467ae5f5a89dddaaab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T21:50:18.314 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:17 smithi097 podman[85350]: 2024-04-26 21:50:17.96076039 +0000 UTC m=+0.495966392 container attach ae299a03b97693afbbe64d06a626c7a1f51dbb392d657467ae5f5a89dddaaab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T21:50:18.661 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 26 21:50:18 smithi097 podman[85350]: 2024-04-26 21:50:18.334752832 +0000 UTC m=+0.869958818 container died ae299a03b97693afbbe64d06a626c7a1f51dbb392d657467ae5f5a89dddaaab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:50:18.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:18 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:18.492+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:18.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:18 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:18.492+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:18.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:18 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:18.492+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:18.891 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:18 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:18.414+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:18.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:18 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:18.414+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:18.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:18 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:18.414+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:19.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T21:50:19.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:19.128 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T21:50:19.129 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T21:50:19.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.10 2024-04-26T21:50:19.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 systemd[1]: Stopping Ceph osd.10 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:19.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.411+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:19.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.411+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:19.642 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.411+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:19.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:19 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:19.521+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:19.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:19 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:19.521+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:19.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:19 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:19.521+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:20.142 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.683+0000 7f32b1200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:20.142 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.683+0000 7f32b1200700 -1 osd.10 93 *** Got signal Terminated *** 2024-04-26T21:50:20.142 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:19 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:19.683+0000 7f32b1200700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:50:20.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:20 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:20.557+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:20.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:20 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:20.557+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:20.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:20 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:20.557+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:20.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:20 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:20.420+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:20.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:20 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:20.420+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:20.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:20 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:20.420+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:21.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:21 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:21.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:21.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:21 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:21.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:21.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:21 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:21.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:21.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:21 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:21.410+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:21.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:21 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:21.410+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:21.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:21 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:21.410+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:22.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:22.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:22.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:22.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:22.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:22.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:22.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:22.518+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:22.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:22 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:22.433+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:22.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:22 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:22.433+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:22.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:22 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:22.433+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:23.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:23 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:23.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:23.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:23 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:23.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:23.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:23 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:23.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:23.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:23 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:23.530+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:23.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:23 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:23.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:23.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:23 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:23.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:23.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:23 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:23.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:23.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:23 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:23.436+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:21.201631+0000 front 2024-04-26T21:49:21.201949+0000 (oldest deadline 2024-04-26T21:50:22.901474+0000) 2024-04-26T21:50:24.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:24 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:24.506+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:24.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:24 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:24.506+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:24.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:24 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:24.506+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:24.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:24 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:24.506+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:24.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:24.449+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:50.892899+0000 front 2024-04-26T21:48:50.892827+0000 (oldest deadline 2024-04-26T21:49:55.592967+0000) 2024-04-26T21:50:24.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:24.449+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:49:02.596355+0000 front 2024-04-26T21:49:02.596445+0000 (oldest deadline 2024-04-26T21:50:05.496574+0000) 2024-04-26T21:50:24.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:24.449+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:08.398386+0000 front 2024-04-26T21:49:08.398580+0000 (oldest deadline 2024-04-26T21:50:13.098446+0000) 2024-04-26T21:50:24.892 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:24 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10[55931]: 2024-04-26T21:50:24.449+0000 7f32aca00700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:21.201631+0000 front 2024-04-26T21:49:21.201949+0000 (oldest deadline 2024-04-26T21:50:22.901474+0000) 2024-04-26T21:50:25.456 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:25 smithi111 podman[66568]: 2024-04-26 21:50:25.180695603 +0000 UTC m=+5.633095647 container died 847b002fca7dadfb6e91eae3fb73cfcfb6875456b639a594b9ce6549939724eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T21:50:25.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:25 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:25.510+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:25.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:25 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:25.510+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:25.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:25 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:25.510+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:25.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:25 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:25.510+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:25.870 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:25 smithi111 podman[66568]: 2024-04-26 21:50:25.576462483 +0000 UTC m=+6.028862550 container cleanup 847b002fca7dadfb6e91eae3fb73cfcfb6875456b639a594b9ce6549939724eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T21:50:25.871 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:25 smithi111 bash[66568]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10 2024-04-26T21:50:26.141 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:25 smithi111 podman[66588]: 2024-04-26 21:50:25.869412037 +0000 UTC m=+0.676509869 container remove 847b002fca7dadfb6e91eae3fb73cfcfb6875456b639a594b9ce6549939724eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:26.715 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:26 smithi111 podman[66675]: 2024-04-26 21:50:26.354939501 +0000 UTC m=+0.039699598 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:26.716 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:26 smithi111 podman[66675]: 2024-04-26 21:50:26.470259202 +0000 UTC m=+0.155019317 container create e597afd4df318320e0e8c81372d97ab93f75249157aaf43d20b4ec082c4eadea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T21:50:26.716 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:26 smithi111 podman[66675]: 2024-04-26 21:50:26.71524109 +0000 UTC m=+0.400001215 container init e597afd4df318320e0e8c81372d97ab93f75249157aaf43d20b4ec082c4eadea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T21:50:26.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:26 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:26.514+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:26.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:26 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:26.514+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:26.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:26 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:26.514+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:26.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:26 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:26.514+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:27.141 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:26 smithi111 podman[66675]: 2024-04-26 21:50:26.7265421 +0000 UTC m=+0.411302221 container start e597afd4df318320e0e8c81372d97ab93f75249157aaf43d20b4ec082c4eadea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:50:27.142 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:26 smithi111 podman[66675]: 2024-04-26 21:50:26.797387624 +0000 UTC m=+0.482147783 container attach e597afd4df318320e0e8c81372d97ab93f75249157aaf43d20b4ec082c4eadea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T21:50:27.523 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 26 21:50:27 smithi111 podman[66675]: 2024-04-26 21:50:27.185805923 +0000 UTC m=+0.870566081 container died e597afd4df318320e0e8c81372d97ab93f75249157aaf43d20b4ec082c4eadea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T21:50:27.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:27 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:27.477+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:27.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:27 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:27.477+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:27.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:27 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:27.477+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:27.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:27 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:27.477+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:27.878 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T21:50:27.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:27.963 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T21:50:27.963 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T21:50:27.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.11 2024-04-26T21:50:28.294 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 systemd[1]: Stopping Ceph osd.11 for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.495+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.495+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.495+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.495+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.589+0000 7f7678e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.589+0000 7f7678e00700 -1 osd.11 93 *** Got signal Terminated *** 2024-04-26T21:50:28.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:28 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:28.589+0000 7f7678e00700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T21:50:29.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:29 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:29.475+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:29.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:29 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:29.475+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:29.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:29 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:29.475+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:29.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:29 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:29.475+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:30.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:30 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:30.483+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:30.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:30 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:30.483+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:30.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:30 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:30.483+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:30.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:30 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:30.483+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:31.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:31 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:31.511+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:31.794 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:31 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:31.511+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:31.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:31 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:31.511+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:31.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:31 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:31.511+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:33.044 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:32.550+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:33.044 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:32.550+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:33.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:32.550+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:33.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:32.550+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:33.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:32 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:32.550+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-26T21:49:27.312987+0000 front 2024-04-26T21:49:27.312448+0000 (oldest deadline 2024-04-26T21:50:32.012188+0000) 2024-04-26T21:50:34.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:33 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:33.577+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-26T21:48:53.403061+0000 front 2024-04-26T21:48:53.402910+0000 (oldest deadline 2024-04-26T21:49:57.503154+0000) 2024-04-26T21:50:34.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:33 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:33.577+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-26T21:48:59.205410+0000 front 2024-04-26T21:48:59.205872+0000 (oldest deadline 2024-04-26T21:50:05.105642+0000) 2024-04-26T21:50:34.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:33 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:33.577+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-04-26T21:49:11.008029+0000 front 2024-04-26T21:49:11.008271+0000 (oldest deadline 2024-04-26T21:50:15.708262+0000) 2024-04-26T21:50:34.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:33 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:33.577+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-26T21:49:15.708926+0000 front 2024-04-26T21:49:15.709053+0000 (oldest deadline 2024-04-26T21:50:22.109877+0000) 2024-04-26T21:50:34.045 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:33 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11[56756]: 2024-04-26T21:50:33.577+0000 7f766f600700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-26T21:49:27.312987+0000 front 2024-04-26T21:49:27.312448+0000 (oldest deadline 2024-04-26T21:50:32.012188+0000) 2024-04-26T21:50:34.423 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:34 smithi143 podman[67449]: 2024-04-26 21:50:34.124724311 +0000 UTC m=+5.693593339 container died 3866a2de8841089eae345d2e981df313edd4dac2ab73e097a3ecd765f3d33dc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:50:34.719 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:34 smithi143 podman[67449]: 2024-04-26 21:50:34.526070193 +0000 UTC m=+6.094939256 container cleanup 3866a2de8841089eae345d2e981df313edd4dac2ab73e097a3ecd765f3d33dc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T21:50:34.719 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:34 smithi143 bash[67449]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11 2024-04-26T21:50:35.044 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:34 smithi143 podman[67469]: 2024-04-26 21:50:34.718422965 +0000 UTC m=+0.583918492 container remove 3866a2de8841089eae345d2e981df313edd4dac2ab73e097a3ecd765f3d33dc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:35.795 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:35 smithi143 podman[67558]: 2024-04-26 21:50:35.436685077 +0000 UTC m=+0.037321745 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:35.796 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:35 smithi143 podman[67558]: 2024-04-26 21:50:35.585873496 +0000 UTC m=+0.186510224 container create 0382d725fdc4fd655b8fc8fae617a9ca738d3511907ee72822be701fe8b5715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T21:50:36.294 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:35 smithi143 podman[67558]: 2024-04-26 21:50:35.872999662 +0000 UTC m=+0.473636374 container init 0382d725fdc4fd655b8fc8fae617a9ca738d3511907ee72822be701fe8b5715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T21:50:36.294 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:35 smithi143 podman[67558]: 2024-04-26 21:50:35.884198218 +0000 UTC m=+0.484834926 container start 0382d725fdc4fd655b8fc8fae617a9ca738d3511907ee72822be701fe8b5715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:36.294 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:36 smithi143 podman[67558]: 2024-04-26 21:50:36.038209416 +0000 UTC m=+0.638846113 container attach 0382d725fdc4fd655b8fc8fae617a9ca738d3511907ee72822be701fe8b5715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T21:50:36.728 INFO:journalctl@ceph.osd.11.smithi143.stdout:Apr 26 21:50:36 smithi143 podman[67558]: 2024-04-26 21:50:36.340597839 +0000 UTC m=+0.941234504 container died 0382d725fdc4fd655b8fc8fae617a9ca738d3511907ee72822be701fe8b5715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:50:37.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T21:50:37.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:37.221 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T21:50:37.222 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T21:50:37.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.a 2024-04-26T21:50:37.564 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:37 smithi097 systemd[1]: Stopping Ceph mds.a for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:38.063 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: 2024-04-26T21:50:37.781+0000 7ff3d8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:38.064 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:37 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a[69822]: 2024-04-26T21:50:37.781+0000 7ff3d8a00700 -1 mds.a *** got signal Terminated *** 2024-04-26T21:50:40.563 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:40 smithi097 podman[85465]: 2024-04-26 21:50:40.278154546 +0000 UTC m=+2.614724877 container died 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T21:50:40.832 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:40 smithi097 podman[85465]: 2024-04-26 21:50:40.690183742 +0000 UTC m=+3.026754073 container cleanup 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:50:40.833 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:40 smithi097 bash[85465]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a 2024-04-26T21:50:41.279 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 26 21:50:40 smithi097 podman[85490]: 2024-04-26 21:50:40.832549994 +0000 UTC m=+0.536668451 container remove 19bd127b4c70f875cc20599c127e850fa76e3c2f107c2605b4dfb924e5b4a8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T21:50:41.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T21:50:41.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:41.421 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T21:50:41.421 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T21:50:41.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.d 2024-04-26T21:50:42.000 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:41 smithi097 systemd[1]: Stopping Ceph mds.d for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:42.313 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d[68746]: 2024-04-26T21:50:41.998+0000 7f0e13000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:42.313 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:42 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d[68746]: 2024-04-26T21:50:41.998+0000 7f0e13000700 -1 mds.d *** got signal Terminated *** 2024-04-26T21:50:43.313 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:43 smithi097 podman[85597]: 2024-04-26 21:50:43.029681103 +0000 UTC m=+1.160679505 container died f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T21:50:43.670 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:43 smithi097 podman[85597]: 2024-04-26 21:50:43.49350172 +0000 UTC m=+1.624500140 container cleanup f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T21:50:43.670 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:43 smithi097 bash[85597]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d 2024-04-26T21:50:44.059 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 26 21:50:43 smithi097 podman[85611]: 2024-04-26 21:50:43.669376055 +0000 UTC m=+0.620797865 container remove f0ba43200f4834f7ab41aca51b4a308a864d32f9df5044123ea543eede4ffd0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:50:44.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T21:50:44.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:44.205 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T21:50:44.205 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T21:50:44.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.g 2024-04-26T21:50:44.786 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:44 smithi097 systemd[1]: Stopping Ceph mds.g for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:45.063 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: 2024-04-26T21:50:44.786+0000 7fb5fc200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:45.063 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:44 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g[69433]: 2024-04-26T21:50:44.786+0000 7fb5fc200700 -1 mds.g *** got signal Terminated *** 2024-04-26T21:50:48.559 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:48 smithi097 podman[85718]: 2024-04-26 21:50:48.286950428 +0000 UTC m=+3.627858726 container died 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:50:48.813 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:48 smithi097 podman[85718]: 2024-04-26 21:50:48.671678177 +0000 UTC m=+4.012586485 container cleanup 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T21:50:48.814 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:48 smithi097 bash[85718]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g 2024-04-26T21:50:49.271 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 26 21:50:48 smithi097 podman[85732]: 2024-04-26 21:50:48.906403136 +0000 UTC m=+0.607906053 container remove 27dc944373712f0386290d68f24292b22cdf495956b94b05222e1f020a2b1b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T21:50:49.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T21:50:49.408 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:49.409 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T21:50:49.409 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T21:50:49.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.j 2024-04-26T21:50:49.563 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:49 smithi097 systemd[1]: Stopping Ceph mds.j for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:50.277 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:49 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j[68369]: 2024-04-26T21:50:49.873+0000 7f525f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:50.277 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:49 smithi097 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j[68369]: 2024-04-26T21:50:49.873+0000 7f525f400700 -1 mds.j *** got signal Terminated *** 2024-04-26T21:50:51.363 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:51 smithi097 podman[85838]: 2024-04-26 21:50:51.09049148 +0000 UTC m=+1.330056830 container died 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:50:51.618 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:51 smithi097 podman[85838]: 2024-04-26 21:50:51.458610661 +0000 UTC m=+1.698176028 container cleanup 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:51.618 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:51 smithi097 bash[85838]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j 2024-04-26T21:50:52.064 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 26 21:50:51 smithi097 podman[85852]: 2024-04-26 21:50:51.617570886 +0000 UTC m=+0.511534769 container remove 7ea633b498b80e2ee886bff34987b121aad8fc6638af6369bdace726c6c5b910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:52.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T21:50:52.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:52.235 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T21:50:52.235 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T21:50:52.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.b 2024-04-26T21:50:52.641 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:52 smithi111 systemd[1]: Stopping Ceph mds.b for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:53.142 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:52 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: 2024-04-26T21:50:52.840+0000 7f751e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:53.142 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:52 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b[61252]: 2024-04-26T21:50:52.840+0000 7f751e800700 -1 mds.b *** got signal Terminated *** 2024-04-26T21:50:56.631 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:56 smithi111 podman[66793]: 2024-04-26 21:50:56.351755483 +0000 UTC m=+3.683440010 container died 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T21:50:56.892 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:56 smithi111 podman[66793]: 2024-04-26 21:50:56.742353574 +0000 UTC m=+4.074038156 container cleanup 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T21:50:56.892 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:56 smithi111 bash[66793]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b 2024-04-26T21:50:57.391 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 26 21:50:56 smithi111 podman[66807]: 2024-04-26 21:50:56.960164822 +0000 UTC m=+0.594881387 container remove 2ff10f2c70fe4481a9bfa6d50d270729fdb6e33a2c9df46e358721150c538b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:50:57.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T21:50:57.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:50:57.629 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T21:50:57.629 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T21:50:57.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.e 2024-04-26T21:50:58.141 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:50:57 smithi111 systemd[1]: Stopping Ceph mds.e for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:50:58.642 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:50:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: 2024-04-26T21:50:58.243+0000 7f2a0b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:58.642 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:50:58 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e[60634]: 2024-04-26T21:50:58.243+0000 7f2a0b600700 -1 mds.e *** got signal Terminated *** 2024-04-26T21:51:00.620 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:51:00 smithi111 podman[66912]: 2024-04-26 21:51:00.329254373 +0000 UTC m=+2.210600848 container died 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T21:51:00.892 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:51:00 smithi111 podman[66912]: 2024-04-26 21:51:00.729565594 +0000 UTC m=+2.610912072 container cleanup 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:51:00.892 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:51:00 smithi111 bash[66912]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e 2024-04-26T21:51:01.391 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 26 21:51:01 smithi111 podman[66936]: 2024-04-26 21:51:01.005900399 +0000 UTC m=+0.660692854 container remove 21578beafa4e8c2e2e88bb3b118ff8e07e15e3291857bf2ea241ffd16c312f6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:51:01.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T21:51:01.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:01.636 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T21:51:01.636 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T21:51:01.636 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.h 2024-04-26T21:51:02.141 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:01 smithi111 systemd[1]: Stopping Ceph mds.h for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:02.641 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: 2024-04-26T21:51:02.230+0000 7fbed8000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:02.642 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:02 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h[60935]: 2024-04-26T21:51:02.230+0000 7fbed8000700 -1 mds.h *** got signal Terminated *** 2024-04-26T21:51:04.622 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:04 smithi111 podman[67045]: 2024-04-26 21:51:04.34107688 +0000 UTC m=+2.252060630 container died 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T21:51:04.892 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:04 smithi111 podman[67045]: 2024-04-26 21:51:04.725071482 +0000 UTC m=+2.636055225 container cleanup 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:51:04.892 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:04 smithi111 bash[67045]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h 2024-04-26T21:51:05.391 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 26 21:51:04 smithi111 podman[67059]: 2024-04-26 21:51:04.951448011 +0000 UTC m=+0.593595395 container remove 2e1384baef5b13b52663dd7e86344e49154b7bbce682545cd7b4cf2083b938c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:51:05.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T21:51:05.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:05.581 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T21:51:05.581 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T21:51:05.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.k 2024-04-26T21:51:05.891 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:05 smithi111 systemd[1]: Stopping Ceph mds.k for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:06.642 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: 2024-04-26T21:51:06.150+0000 7f3baec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:06.642 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:06 smithi111 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k[60322]: 2024-04-26T21:51:06.150+0000 7f3baec00700 -1 mds.k *** got signal Terminated *** 2024-04-26T21:51:08.633 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:08 smithi111 podman[67164]: 2024-04-26 21:51:08.348244556 +0000 UTC m=+2.328373348 container died 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T21:51:08.891 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:08 smithi111 podman[67164]: 2024-04-26 21:51:08.753807663 +0000 UTC m=+2.733936478 container cleanup 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-26T21:51:08.892 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:08 smithi111 bash[67164]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k 2024-04-26T21:51:09.330 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 26 21:51:09 smithi111 podman[67178]: 2024-04-26 21:51:09.071747378 +0000 UTC m=+0.706408620 container remove 45e7366b0eb8371398bbb8461acd4257cf00d90ff4e4e36d17793b7311a62f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T21:51:09.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T21:51:09.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:09.449 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T21:51:09.449 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T21:51:09.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.c 2024-04-26T21:51:09.794 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:09 smithi143 systemd[1]: Stopping Ceph mds.c for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:10.294 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:09 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: 2024-04-26T21:51:09.989+0000 7f570d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:10.294 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:09 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c[61779]: 2024-04-26T21:51:09.989+0000 7f570d800700 -1 mds.c *** got signal Terminated *** 2024-04-26T21:51:12.713 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:12 smithi143 podman[67679]: 2024-04-26 21:51:12.355631646 +0000 UTC m=+2.487325586 container died e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T21:51:13.044 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:12 smithi143 podman[67679]: 2024-04-26 21:51:12.833589214 +0000 UTC m=+2.965283150 container cleanup e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T21:51:13.045 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:12 smithi143 bash[67679]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c 2024-04-26T21:51:13.498 INFO:journalctl@ceph.mds.c.smithi143.stdout:Apr 26 21:51:13 smithi143 podman[67693]: 2024-04-26 21:51:13.051210115 +0000 UTC m=+0.678149319 container remove e32b9af4707125c0fb577da49ca2f4147ba0b0134f357f23e281030402edaca3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:51:13.537 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T21:51:13.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:13.645 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T21:51:13.645 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T21:51:13.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.f 2024-04-26T21:51:13.794 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:13 smithi143 systemd[1]: Stopping Ceph mds.f for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:14.794 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:14 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: 2024-04-26T21:51:14.309+0000 7f8cb8600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:14.794 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:14 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f[61480]: 2024-04-26T21:51:14.309+0000 7f8cb8600700 -1 mds.f *** got signal Terminated *** 2024-04-26T21:51:16.675 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:16 smithi143 podman[67798]: 2024-04-26 21:51:16.352527639 +0000 UTC m=+2.292615172 container died cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:51:17.044 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:16 smithi143 podman[67798]: 2024-04-26 21:51:16.787031799 +0000 UTC m=+2.727119326 container cleanup cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T21:51:17.045 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:16 smithi143 bash[67798]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f 2024-04-26T21:51:17.512 INFO:journalctl@ceph.mds.f.smithi143.stdout:Apr 26 21:51:17 smithi143 podman[67812]: 2024-04-26 21:51:17.063307347 +0000 UTC m=+0.696345829 container remove cdfc7e6d0e299e48e9d79d27a4c3a144c48d23a4dd960e13381686058ed58d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:51:17.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T21:51:17.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:17.655 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T21:51:17.655 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T21:51:17.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.i 2024-04-26T21:51:18.214 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:17 smithi143 systemd[1]: Stopping Ceph mds.i for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:18.544 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:18 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: 2024-04-26T21:51:18.213+0000 7f7752e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:18.545 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:18 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i[62092]: 2024-04-26T21:51:18.213+0000 7f7752e00700 -1 mds.i *** got signal Terminated *** 2024-04-26T21:51:20.686 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:20 smithi143 podman[67916]: 2024-04-26 21:51:20.354457023 +0000 UTC m=+2.285523721 container died a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T21:51:21.033 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:20 smithi143 podman[67916]: 2024-04-26 21:51:20.837496 +0000 UTC m=+2.768562744 container cleanup a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:51:21.033 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:20 smithi143 bash[67916]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i 2024-04-26T21:51:21.294 INFO:journalctl@ceph.mds.i.smithi143.stdout:Apr 26 21:51:21 smithi143 podman[67930]: 2024-04-26 21:51:21.03273216 +0000 UTC m=+0.665880608 container remove a3f36c40fc85d44654717a9c9442962bedd1299da2b43064d2cc0e953ed6b546 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:51:21.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T21:51:21.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:21.673 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T21:51:21.674 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T21:51:21.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.l 2024-04-26T21:51:22.044 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:21 smithi143 systemd[1]: Stopping Ceph mds.l for 9ce681f8-0412-11ef-bc93-c7b262605968... 2024-04-26T21:51:22.544 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: 2024-04-26T21:51:22.224+0000 7f2aa5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:51:22.544 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:22 smithi143 ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l[61159]: 2024-04-26T21:51:22.224+0000 7f2aa5600700 -1 mds.l *** got signal Terminated *** 2024-04-26T21:51:24.631 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:24 smithi143 podman[68035]: 2024-04-26 21:51:24.352897031 +0000 UTC m=+2.251887693 container died 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:51:25.019 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:24 smithi143 podman[68035]: 2024-04-26 21:51:24.768505721 +0000 UTC m=+2.667496376 container cleanup 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:51:25.020 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:24 smithi143 bash[68035]: ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l 2024-04-26T21:51:25.294 INFO:journalctl@ceph.mds.l.smithi143.stdout:Apr 26 21:51:25 smithi143 podman[68049]: 2024-04-26 21:51:25.019131718 +0000 UTC m=+0.653956333 container remove 0a58b5498d0c411869fdd7627e6cb3b22241bf9c95c3ef1dee18aa8268e0b1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-9ce681f8-0412-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T21:51:25.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ce681f8-0412-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T21:51:25.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T21:51:25.641 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T21:51:25.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:51:25.961 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:30.930 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:51:31.244 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:36.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T21:51:36.354 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:41.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:51:41.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:51:41.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:51:41.630 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T21:51:41.633 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi097/crash 2024-04-26T21:51:41.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:51:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:51:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:51:41.733 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi111/crash 2024-04-26T21:51:41.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:51:41.858 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:51:41.858 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:51:41.864 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi143/crash 2024-04-26T21:51:41.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash -- . 2024-04-26T21:51:41.960 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T21:51:41.961 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-04-26T21:51:41.964 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T21:51:41.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T21:51:42.073 INFO:tasks.cephadm:Compressing logs... 2024-04-26T21:51:42.074 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:51:42.078 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:51:42.085 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:51:42.133 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:51:42.133 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-26T21:51:42.133 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:51:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:51:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T21:51:42.136 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T21:51:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T21:51:42.138 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:51:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.j.log: 90.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:51:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T21:51:42.140 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T21:51:42.141 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.3.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T21:51:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T21:51:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T21:51:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:51:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T21:51:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:51:42.151 INFO:teuthology.orchestra.run.smithi097.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-26T21:51:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:51:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T21:51:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T21:51:42.157 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:51:42.158 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:51:42.158 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:51:42.160 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T21:51:42.160 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T21:51:42.162 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T21:51:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T21:51:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:51:42.164 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T21:51:42.164 INFO:teuthology.orchestra.run.smithi111.stderr: 90.4% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:51:42.164 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:51:42.165 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T21:51:42.166 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T21:51:42.167 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T21:51:42.169 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:51:42.170 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T21:51:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log 2024-04-26T21:51:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T21:51:42.172 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T21:51:42.174 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T21:51:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:51:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T21:51:42.175 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T21:51:42.175 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T21:51:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:51:42.176 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T21:51:42.177 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T21:51:42.178 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T21:51:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T21:51:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T21:51:42.180 INFO:teuthology.orchestra.run.smithi111.stderr: 93.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T21:51:42.180 INFO:teuthology.orchestra.run.smithi143.stderr: 90.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T21:51:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:51:42.181 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T21:51:42.183 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T21:51:42.186 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T21:51:42.187 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:51:42.188 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:51:42.189 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.11.log: 82.4% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T21:51:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T21:51:42.191 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T21:51:42.192 INFO:teuthology.orchestra.run.smithi143.stderr: 93.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T21:51:42.192 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:51:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.8.log: /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T21:51:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T21:51:42.214 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T21:51:42.214 INFO:teuthology.orchestra.run.smithi111.stderr: 90.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:51:42.215 INFO:teuthology.orchestra.run.smithi143.stderr: 90.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T21:51:42.226 INFO:teuthology.orchestra.run.smithi097.stderr: 94.6% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T21:51:42.239 INFO:teuthology.orchestra.run.smithi111.stderr: 93.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T21:51:42.248 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T21:51:42.252 INFO:teuthology.orchestra.run.smithi143.stderr: 94.0% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T21:51:42.256 INFO:teuthology.orchestra.run.smithi111.stderr: 93.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T21:51:42.262 INFO:teuthology.orchestra.run.smithi111.stderr: 93.9% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T21:51:42.270 INFO:teuthology.orchestra.run.smithi143.stderr: 94.4% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T21:51:42.273 INFO:teuthology.orchestra.run.smithi143.stderr: 94.7% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T21:51:42.302 INFO:teuthology.orchestra.run.smithi143.stderr: 94.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T21:51:42.413 INFO:teuthology.orchestra.run.smithi111.stderr: 94.0% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T21:51:42.531 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T21:51:42.619 INFO:teuthology.orchestra.run.smithi143.stderr: 92.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T21:51:42.627 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T21:51:42.633 INFO:teuthology.orchestra.run.smithi143.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T21:51:42.649 INFO:teuthology.orchestra.run.smithi111.stderr: 92.4% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T21:51:42.660 INFO:teuthology.orchestra.run.smithi111.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T21:51:42.707 INFO:teuthology.orchestra.run.smithi097.stderr: 92.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T21:51:42.730 INFO:teuthology.orchestra.run.smithi143.stderr: 92.0% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T21:51:42.731 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T21:51:42.735 INFO:teuthology.orchestra.run.smithi111.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T21:51:42.788 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T21:51:42.812 INFO:teuthology.orchestra.run.smithi111.stderr: 92.4% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T21:51:42.837 INFO:teuthology.orchestra.run.smithi143.stderr: 92.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T21:51:42.893 INFO:teuthology.orchestra.run.smithi143.stderr: 92.2% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T21:51:42.896 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-04-26T21:51:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m0.757s 2024-04-26T21:51:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m2.893s 2024-04-26T21:51:42.897 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m0.310s 2024-04-26T21:51:42.976 INFO:teuthology.orchestra.run.smithi111.stderr: 92.7% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T21:51:42.980 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-04-26T21:51:42.980 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m0.851s 2024-04-26T21:51:42.980 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m3.187s 2024-04-26T21:51:42.980 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.337s 2024-04-26T21:51:43.164 INFO:teuthology.orchestra.run.smithi097.stderr: 90.3% -- replaced with /var/log/ceph/9ce681f8-0412-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T21:51:43.169 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-26T21:51:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m1.060s 2024-04-26T21:51:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m3.610s 2024-04-26T21:51:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.366s 2024-04-26T21:51:43.171 INFO:tasks.cephadm:Archiving logs... 2024-04-26T21:51:43.171 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi097/log 2024-04-26T21:51:43.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:51:43.831 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi111/log 2024-04-26T21:51:43.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:51:44.497 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi143/log 2024-04-26T21:51:44.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:51:44.972 INFO:tasks.cephadm:Removing cluster... 2024-04-26T21:51:44.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force 2024-04-26T21:51:45.275 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:45.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force 2024-04-26T21:51:46.206 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:46.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ce681f8-0412-11ef-bc93-c7b262605968 --force 2024-04-26T21:51:47.142 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 9ce681f8-0412-11ef-bc93-c7b262605968 2024-04-26T21:51:47.767 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T21:51:47.768 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:51:47.825 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:51:47.883 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T21:51:47.941 INFO:tasks.cephadm:Teardown complete 2024-04-26T21:51:47.941 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:51:47.950 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:51:47.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:51:47.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:51:47.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:51:48.125 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:51:48.134 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:51:48.135 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:51:48.142 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:51:48.150 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:51:48.190 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:51:48.197 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:51:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:51:48.200 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-26T21:51:48.201 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +1403us[+1300us] +/- 52ms 2024-04-26T21:51:48.201 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -473us[ -473us] +/- 57ms 2024-04-26T21:51:48.201 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -1483us[-1483us] +/- 73ms 2024-04-26T21:51:48.201 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:51:48.203 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:51:48.207 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:51:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-26T21:51:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1343us[+1277us] +/- 52ms 2024-04-26T21:51:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -540us[ -540us] +/- 57ms 2024-04-26T21:51:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -1546us[-1546us] +/- 73ms 2024-04-26T21:51:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1294us[+1289us] +/- 52ms 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -570us[ -576us] +/- 57ms 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -1589us[-1595us] +/- 73ms 2024-04-26T21:51:48.214 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:51:48.215 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:51:48.225 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:51:48.226 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:51:48.235 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:51:48.433 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:51:48.610 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:51:48.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:51:48.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:48.874 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:51:48.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:48.967 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:51:49.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:49.061 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:51:49.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:51:49.079 INFO:teuthology.task.internal:Duration was 2426.424211 seconds 2024-04-26T21:51:49.079 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:51:49.088 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:51:49.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:51:49.096 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:51:49.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:51:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:51:49.242 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:51:49.246 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:51:49.738 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:51:49.738 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T21:51:49.738 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:51:49.812 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-04-26T21:51:49.812 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:51:49.887 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-04-26T21:51:49.888 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:51:49.964 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:51:49.965 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:51:49.972 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:51:49.980 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:51:50.064 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:51:50.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:51:50.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:51:50.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:51:50.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:51:50.547 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:51:50.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:51:50.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:51:50.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:51:50.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:51:50.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:50.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:50.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-26T21:51:50.933 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-04-26T21:51:50.940 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-04-26T21:51:51.034 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:51.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:51.089 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:51.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:51.143 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:51:51.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:51:51.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:51:51.209 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:51:51.210 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi097 2024-04-26T21:51:51.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:51:51.314 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi111 2024-04-26T21:51:51.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:51:51.411 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206/remote/smithi143 2024-04-26T21:51:51.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:51:51.510 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:51:51.510 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:51:51.518 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:51:51.525 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:51:51.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:51:51.593 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:51:51.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:51:51.604 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:51:51.604 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:51:51.610 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:51:51.615 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:51:51.658 INFO:teuthology.orchestra.run.smithi097.stdout: 393241 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:51 /home/ubuntu/cephtest 2024-04-26T21:51:51.664 INFO:teuthology.orchestra.run.smithi111.stdout: 393560 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:51 /home/ubuntu/cephtest 2024-04-26T21:51:51.670 INFO:teuthology.orchestra.run.smithi143.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:51 /home/ubuntu/cephtest 2024-04-26T21:51:51.674 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:51:51.685 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:51:51.705 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T21:51:51.731 DEBUG:teuthology.nuke:shortname: smithi097 2024-04-26T21:51:51.732 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:51:51.757 DEBUG:teuthology.nuke:shortname: smithi111 2024-04-26T21:51:51.757 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:51:51.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.289075', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhef9fIanZyyV04PnAodD9TchsY1YKiHpcf6kEUBcPL+SWPNqmF2wNXeom1al4IEBjytX0a8lhXC8BGs3mQRL4='} 2024-04-26T21:51:51.781 DEBUG:teuthology.nuke:shortname: smithi143 2024-04-26T21:51:51.781 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:51:51.784 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.285440', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfTqABpds1YpiUBt1a3TRkE0TRIUCc+Jfg9fL1sHFkOCexV+PaFb1JsKJgTPkUkhHWIJ+F4pRJyybjC4Amrjog='} 2024-04-26T21:51:51.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:01:51.287480', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6T4C3cqgJiptYm9ScPY5OGgfiouzKH3nMH6DZwiitrHp5vaFGjXZ6naTAm8/fTs0cppSQC8p4PSX2pL/aCgUA='} 2024-04-26T21:51:51.818 INFO:teuthology.orchestra.console.smithi097:Power off 2024-04-26T21:51:51.818 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T21:51:51.840 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-04-26T21:51:51.841 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:51.861 INFO:teuthology.orchestra.console.smithi111:Power off 2024-04-26T21:51:51.862 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T21:51:51.872 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-26T21:51:51.884 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2024-04-26T21:51:51.884 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:51.902 INFO:teuthology.orchestra.console.smithi143:Power off 2024-04-26T21:51:51.902 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T21:51:51.911 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-26T21:51:51.924 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2024-04-26T21:51:51.924 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:51.947 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-04-26T21:51:55.874 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:55.912 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:55.949 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:51:55.984 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-26T21:51:56.022 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-26T21:51:56.058 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-04-26T21:51:59.987 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:00.022 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:00.059 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:00.100 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-26T21:52:00.133 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-26T21:52:00.169 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-04-26T21:52:04.102 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:04.135 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:04.170 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T21:52:04.213 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-04-26T21:52:04.243 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2024-04-26T21:52:04.280 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-04-26T21:52:04.314 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-04-26T21:52:04.344 INFO:teuthology.orchestra.console.smithi111:Power off completed 2024-04-26T21:52:04.381 INFO:teuthology.orchestra.console.smithi143:Power off completed 2024-04-26T21:52:04.487 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} duration: 2426.4242107868195 failure_reason: 'Command failed on smithi097 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6bcf2f8bdb994f7ab96583a11f36f637 status: fail success: false 2024-04-26T21:52:04.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:52:04.575 INFO:teuthology.run:FAIL